Why does nobody talk about AI bias in hiring tools?

I was building a resume screener using OpenAI's API and a former colleague pointed out my training data was 80% male applicants from tech. That feedback hit hard because I hadn't thought about how my own choices were baking in bias. Now I'm wondering if we should be more upfront about the data limits in these tools before companies use them. Has anyone else gotten feedback that forced you to rethink your AI project from scratch?

3 comments

3 Comments

the_lucas26d agoMost Upvoted

Went straight from "this will save HR so much time" to "oh no I'm the reason software discriminates" real quick.

grant.nina26d agoTop Commenter

And THATS the part nobody wants to sit with right? @the_lucas nailed it because we all start thinking we're helping and then realize we're just repeating old problems. I had a friend who built a customer service bot and fed it all their old support tickets and turns out those tickets were mostly from angry dudes who yell at reps. So the bot started being a total jerk to women and nonbinary callers. The company was blindsided like wait we just used our own data? Yeah exactly your data was the problem from day one.

piperwhite15d ago

Yeah that part about "we just used our own data" really gets me. It's like people think data is this neutral thing and not a bunch of real interactions with all their baggage and bias baked in. I mean, of course a bot trained on angry customers is gonna come out angry, that's just garbage in garbage out but with hurt feelings attached. It's wild how companies will spend millions on shiny tools without ever stopping to ask if the foundation is rotten. Solidarity to your friend for being the one to figure it out the hard way though.