you-tldr.com Summary:
The speaker, the co-founder and CEO of Ultranauts, talks about how they have built a team that is spread out across 29 states in the US, which is predominantly made up of neurodiverse individuals, especially those on the autism spectrum. They believe that neurodiversity can be a competitive advantage for businesses and that it is crucial to create a truly inclusive workplace in order to achieve it. They have reimagined every aspect of their business, creating a universal workplace that applies universal design principles to the system of work. They are focusing on transparency in decision-making processes and on the well-being of their employees, as well as innovating their practices with tools such as the BioDex, a self-authored quick start guide for how to work with each individual. Ultranauts has demonstrated that they can deliver better value for their clients through automated testing for software and data quality engineering. Their team is less lonely than the typical American worker, and they have shown that their workplace practices can be beneficial for all team members.
The speaker discussed the importance of diversity and changing the recruitment pipeline, citing the limitations of traditional recruiting tools like resumes and interviews. They suggested using more objective ways of assessing talent, such as job tests, and investing in different sourcing methods to attract a wider range of applicants. The speaker also emphasized using evidence to inform practices and being willing to move past familiar methods. They acknowledged the importance of data in decision-making but cautioned against calcifying biases into future AI projects. The speaker recognized the need for more research and development in building interpretable and explainable AI models.
Full Transcript:
it's a pleasure to be here
0:15um welcome everyone again my name and as matt is i am the director for diversity equity inclusion at columbia business school it's my pleasure to um to to welcome you here to the nora diversity and untapped competitive advantage session
0:30so if you didn't know that is where you are at and we are happy to have you uh and i'm also pleased to introduce you um uh as matt mentioned he's the co-founder and ceo of ultranox and we have about 30 minutes uh together
0:45to learn just about how we can reimagine the way an organization can hire and manage talent including some universal workplace practices that employers can adopt to be a more inclusive organization it's great to be here i'm going to share
1:00my screen and do a little bit of a presentation and then we'll have more of a conversation uh with pamela all right so ultranauts has been around for eight years my
1:15co-founder art checkman and i started the company with a simple mission to demonstrate that neurodiversity could be a competitive advantage for business and our theory of change was simply to
1:30build a world-class commercially viable business that could deliver better value for our clients and along the way reimagine how a company hires talent manages teams develops
1:45careers and operates so that a much wider profile of talent and a much wider group of humans could work together and collaborate and thrive and take everything we're learning and
2:00share it uh with other employers so that the journey to towards neurodiversity and and embracing diversity in in all its forms could be a little bit easier uh for those who come after us so i'm always
2:15happy to share what we've been up to and what we're learning so today we are a team that's spread out over 29 states across the u.s we're all on shore and three-quarters of our team across
2:30the company are on the autism spectrum and we have autistic colleagues not just our quality engineers and quality analysts which is most people at ultranauts because we're a quality engineering firm but also on our hr recruiting team or on
2:45the leadership team and this is by design because we fundamentally believe that by bringing together different brain types different information processing models different problem solving styles and forging
3:00collaborative teams we could do better we could solve more complex problems we could surface more unique insights and just drive improvement in a more continuous way and in our case all through the lens of
3:15quality of software data and and business processes and while we are of course neurodiverse we also are diverse along many other dimensions um forty percent of our team are cisgender
3:30female five percent of our team are non-binary five percent are trans 12 percent have other gender identities and so we may be the most gender diverse engineering firm out there um 28 of our team are
3:45people of color and then not just in terms of demographics but we also think about diversity in terms of social economics three-quarters of our team were unemployed or underemployed uh when they joined the firm 40 percent
4:00of our team used to live below the the poverty income line before they join the firm and and so as we think about diversity we of course have a mission that's anchored in neurodiversity
4:15but we know that in order to achieve neurodiversity there are no point solutions and we have to create a truly inclusive workplace and in order to do that we've had to reimagine every aspect of the business and we are on this journey
4:30to creating what we call a universal workplace which simply means applying universal design principles to the system that is work and that means building flexibility in as the norm versus a accommodation or an
4:45exception so everybody works from home we've been that way for eight years um that doesn't work for everyone but certainly for most astronauts um many may not have even applied if they thought they would be forced to commute into a
5:00you know same space and and work in a typical office environment but we think of flexibility in many other dimensions and we'll talk a bit more about that um we have moved away from the notion of an fte a full-time equivalent as the only unit of work that people need to
5:15fit into we think of it more in terms of a dte a desired time equivalent like what is optimally productive for you is it quote full time or is it three quarters time because if you're hyper productive for part of the week there's no reason you can't
5:30have a salary and progress in your career transparency in particular around decision making is absolutely critical as you bring together people from very different experiences and backgrounds and interests and of course focusing on well-being we
5:45have a bot that measures pulls our team at 5 pm local time every day each day is a single pole we cycle through about a dozen polls which tie back to dimensions of inclusion and well-being that we as a
6:00group have decided are important to us um one of them is loneliness and we've been measuring loneliness as a team for many years now we're about to start measuring psychological safety through an independent
6:15nonprofit group that's going to audit our team our company on psychological safety and those results would be published internally publicly and that creates extreme accountability because it's not just an hr team's problem with the leadership
6:30team's problem if there are problems we all need to first own up to where we need to be better and then actually take take actions um to move us towards a place where everyone is feeling included and productive and
6:45creating value and being valued and maybe most importantly for us inclusion is more than a feeling or a concept we're a bunch of engineers so we've designed inclusion into our core business practices
7:00in terms of how we think about recruiting in a more objective way or how we run our projects which are all agile and scrum in a more inclusive way or how we think about training and upskilling and designing instructure material and
7:15technical training in a more inclusive way and as a result of all of that even though we are fully distributed everyone works from home in 29 states uh most of us have never met each other and we're incredibly diverse you know on
7:30any of our teams we'll have a wide range of learning styles and um information processing models and communication preferences and stress triggers we might have colleagues who have ptsd or severe anxiety with all of that
7:45because we measure loneliness we know this our team is a whole lot less lonely than the typical american worker and before covid 40 of americans reported feeling lonely at work and
8:00arguably that's a lot worse now and so we've been able to create a connected engaged team in uh in in a very different and maybe more
8:15extreme circumstance but it's because we have to design for sort of an extreme user of the system that is work that we've made a workplace that's better for everyone and because of that we've shown over and over again that we can deliver better value for our
8:30clients in our core work which is software quality engineering data quality engineering will partner with a client's internal software development team or engineering team and build automated testing to make sure the software they're building works the way it's supposed to or will partner
8:45with an internal data science team or data analytics team to do an enterprise-wide data quality audit to figure out what's good data you can trust or validate the output of your machine learning model so you can make good decisions and it's fairly technical and complex work we're of course competing with lots
9:00of large consultancies but over and over again we've demonstrated that we could deliver dramatically better results to a wide range of fortune 5 firms
9:15the likes of you know aig or berkshire hathaway bloomberg bny mellon colgate cigna and so on and we of course work with pure tech companies like slack as well
9:30and outside of our core business we've also been able to innovate both of our our own practices that we're now starting to share with the world and as well as uh open up adjacencies that um aren't in our core quality engineering business
9:45i mean i'll share a couple examples of that you know a few years ago a teammate at alternate said you know you never figure out how to work with some of the teammates on this project i just wish humans came with a user manual and of course that's a simple and
10:00powerful idea and from that simple thought emerge what we now call the biodex we've built it into a slackbot and it's a self-authored quick start guide for how to work with me it's very pragmatic it's not theoretical like like a myers-briggs kind of
10:15framework it has 20 fields including some that are absolutely critical to know when you're working on a distributed and diverse team like what are your feedback preferences
10:30because while most managers are taught to for example give critical feedback in the moment in a live conversation sandwiched by positive and affirming comments turns out that's actually doesn't work for many humans it's um there's no silver bullet and so
10:45at ultranauts all i need to do is look up someone's feedback preferences it's a single command in slack and i know whether for example if i have some constructive feedback to give do they want it in the moment or and these are drop down menus so it's fairly standardized you know or at the
11:00end of the day or end of the week um uh i know if how they'd like to receive it is it on a call or would they prefer it in writing and then have a conversation if something's not clear or they don't agree and framing
11:15you know because we all get defensive we're human we don't like being criticized um is there a way i could phrase or frame this conversation to simply make it easier to hear and if you're thinking at this point gosh my team could use that yes
11:30this is not about you know what neurodivergent team members need this is what all team members need and it just so happens that if you have colleagues who might be triggered having a simple tool like this can prevent those extreme events from
11:45happening we've also re-um sort of worked the way we run our work and again because we're an engineering team everything we do is
12:00using agile and scrum and we've authored um what we call inclusive agile we just published in a peer review journal a few months ago and we're sharing this more broadly because we think there's agile is fantastic and so is scrum and it can be adapted
12:15so that it's fantastic for everyone and not just some um sort of theoretical mean of person that doesn't exist and then outside of our core business we've just launched a practice that
12:30takes all of our capabilities around data quality engineering you know structuring data analyzing data surfacing patterns of anomalies and data and so on and applying all of that to the data exhaust that's generated
12:45along around the employee experience uh to spot patterns of bias we call it talent bias detection and this has been so well received that our teams are actually booked through the end of the year it's highly specialized work and so if you need talent by detection
13:00we'll see you in q1 2022 but essentially we're able to go in and go beyond theory and hypotheses and good intentions to say okay let's rev let's actually audit your performance review process all of your performance review data
13:15with statistical analysis and nlp and actually precisely pinpoint where there are patterns of eyes and all of that allows us as a business because we are a
13:30business we're a for-profit enterprise to continue to grow and evolve and we've been growing our top line at over 50 a year and last year while q2 was rough for us as it was for many organizations we took a real hit
13:45um to the business we recovered by q3 and ended the year having grown 70 percent because we have fundamentally created an organization and an environment where we have incredibly capable humans who are able to use their unique
14:00strengths and not leave any human potential on the table which then allows us to do better than most others out there and that's it
14:15thank you so much i just this was amazing we have so many questions that have come into the chat um i think that what i will do is maybe i'll just start off with a couple of questions
14:30of my own and then we'll we'll open up to some of the wonderful questions that are in the q a um first i want to say that i just love that you um really highlight that the work that you do actually it it there's a return on your
14:45investment you know you invest in people and and doing really ethical um business practices and as a result of that that you've seen an increase and you've seen the consistent growth and that's um that's a powerful
15:00statement and to show the importance of what you do is also very beneficial to the work into your organization's um success right so some of the questions in the chat kind of allude to this it's about just
15:15you know diversity and not you know not just getting people through the pipeline but just kind of really changing the pipeline itself and how we look at it and so um to me you know being one of the more diverse companies out there both in gender you talk about closing um the poverty
15:30gap and obviously of course you know employing more diverse employees and also being diverse in how you actually imagine the workplace with flexible workplace norms and
15:45really integrating just all this stuff into your core business practices um what do you think about some other organizations that are using data analytics and applying neurodiversity to develop job placement technologies for helping businesses
16:00clarify job roles in higher diverse talent um you know so i think the there is a need to be more evidence-based
16:15and objective um and a willingness to uh move beyond what's comfortable right so um if we think of sort of the typical
16:30recruiting toolkit you look at a resume do an interview or two or five and then you make a hire and the success rates of this approach are not that great there's been a ton of work done
16:45looking at um the efficacy of those common recruiting tools um there have been more than one meta-analyses published um that looked at the correlation between like years of work experience and on the job performance or you know interview assessment and
17:00on-the-job performance and it turns out for example that the correlation between years of work experience and on-the-job performance is like an r squared of 0.03 or something something absurd like in an r squared is simply the you know measure of correlation zero means no correlation one is a
17:15perfect correlation there's no correlation and so when we try to pattern match the past we leave out some really important considerations one is uh what if someone who's
17:30incredibly capable hasn't had a fair shot to access opportunity that means they'll never access opportunity because they don't have the experience they need you know um
17:45and uh and so i would say the more we can use evidence to inform our practices and be willing to move past what's familiar for example we all like doing interviews we think you know we overestimate our ability to spot talent and it turns out at least
18:00most studies would look at interview assessments if you take unstructured interviews which are terrible and i can't believe these still happen you know r squared of maybe point one structured interview where you have the same set of questions and a scoring rubric
18:15that determines what good answers are before the conversation bounds the very natural human bias that kicks in we just have to assume that bias training doesn't get rid of bias and you need the environment
18:30that you're operating within to have nudges and tools that constrain that bias and so all this to say step one is you know of course structured interviews are helpful but have more objective ways to actually
18:45assess talent for example job tests and these are not just for entry-level roles or technical roles at ultranauts we use job tests for recruiting period it's not a accommodation it's not a you know special program for uh female applicants or autistic
19:00applicants because job tests are more objective and so every job at alternates has a job test including entry-level quality analysts we hired our head of sales to join the
19:15leadership team to take a job test because it's a better way to work all applicants not just marginalized applicants just all applicants because it's a better way to recruit the second i would say is um sourcing matters
19:30so while our recruiting kind of talent screening process is the same for everyone if you get a job at alternates it's because you have the right brain for the right job we don't care who you are we embrace who
19:45you are but that doesn't make you you know um have a higher or lower likelihood of getting the job but where we do invest differentially is how we attract reach and meet applicants potential
20:00applicants where they are we go beyond universities so we did a recent informal look at like if we look at our top performers what are sort of the common traits we're engineers we're looking for patterns
20:15and it turns out um you know almost a third of that group don't have a college degree right so it doesn't matter it doesn't mean you are going to be better at this job and so
20:30we don't limit ourselves to degrees or certifications we value those but those are not requirements we don't limit ourselves to people who might be on linkedin or zip recruiter we use those platforms but we don't limit ourselves to it because there are
20:45lots of talent who don't may not feel comfortable or may not have the confidence or may not even know um how to navigate those systems we don't uh limit ourselves to like
21:00job placement groups or other kind of focused platforms because there may be people for example who are neurodivergent do not have a formal diagnosis or are uncomfortable seeking out help and so but could be a fantastic quality
21:15engineer and we don't want to miss out on that talent and so we think of sourcing more like grassroots outreach and content marketing in order to meet great talent where they are and we have to try harder to cast as
21:30wide a net as possible so that we don't end up with um you know unintentionally a um a narrower group than humans because we want we want humans
21:45i would say you know when it comes to um analytics again the there's of course a role for data and objective decisions to to improve the process um and our entire talent bias
22:00kind of initiative is is about that with the caution that in order to train a model you need history and if the history is biased and narrow then you have to try extra hard to make
22:15sure that's not repeated in the future you're creating excellent um so pretty much what what i'm hearing is that data
22:30is important in you and you you also you use it in in your practices but there is there is a caution about it right that you have to be mindful and it's i think the you know we we um uh
22:45kind of do quality uh checks and build automation to validate the output of analytics engines and machine learning models so we're in this domain like we um have a decent understanding of
23:00where these systems and tools can be useful versus not and really the the big um watch out is to make sure that we are not calcifying the past into
23:15systems that are making predictions or driving human decisions right which is why you know obviously you mentioned just the importance of pairing it with grassroots outreach right and going beyond just what we
23:30what we may find on some of our um you know linkedin or some of those platforms that you know we typically just kind of like um benchmark to or just that's our default right but that we actually expanded and be a lot more intentional um i'm just highlighting that because we do have some questions that that were
23:45asked about how do you find employees um and and and clients and and them re-entering the workplace and i think that you've already you know touched on that in a number of areas so i think maybe i'll just see if there's
24:00there was another question that came in about um you know the ways that you um use data so um how companies are looking at data and the potential environmental impact of their future ai
24:15projects um koran is asking how are companies doing that how are they looking at their data and its impact on future ai projects and is that a best practice being promoted when in evaluating the app um that's a
24:30kind of big question and i'm certainly not you know this is where i would turn over to my colleague nicole ratsville who's our principal data scientist and head of quality
24:45who's just you know um really a thought leader in the space around quality of systems but the thing i would say is there's a lot of noise around digital transformation and initially it was around creating digital
25:00experiences for consumers and it's moved fairly quickly to um having better decision making and predictions um
25:15built on on top of ai and so the sort of very um boring challenge that a lot of data science teams are facing is that um you're trying to build sophisticated models on top of garbage because you've got just
25:30not great data sets and in some cases just massive amounts of third-party data streaming in or mountains of dark data inside this enterprise silo everywhere that you don't actually know what you can trust versus not so that's just at a very basic level and part of
25:45the reason we you know exists and and are growing um i think we're still we need to um sort of clean up those basic issues
26:00as we're tackling the more interesting ones like bias training data sets and there's research happening now some of it is getting to industry
26:15in terms of building interpretable machine learning models or explainable ai so that a human can interact and understand how certain decisions were made and that's a key component of then being able to say well is this thing working
26:30the way it's supposed to or not but i think we're still a few years away from um sort of interpretability being designed into the
26:45systems that are being deployed commercially
![Neurodiversity and Leadership in the Hiring Process [draft]](https://static.wixstatic.com/media/0a19fd_a3299cad5f2041349ac65e30de6ead87~mv2.png/v1/fill/w_522,h_392,fp_0.50_0.50,q_95,enc_auto/0a19fd_a3299cad5f2041349ac65e30de6ead87~mv2.png)
Comments