Thinking about pursuing a career in ML and AI? Here’s what you need to know.
By Scott Matteson | April 26, 2019, 10:15 AM PST
Staying ahead of the tide is the mantra for today’s technology professionals. As technology and related processes evolve, those who work in the field must update their skills and even careers if necessary.
Some traditional help desk, system, and network administrator roles are fading out to be replaced by endeavors requiring a heftier and more diverse skills set. Machine learning (ML) and artificial intelligence (AL) are two such fields making steady inroads into the IT world. People looking for a future career in technology would do well to become familiar with both ML and AI.
I spoke to Dillon Erb, CEO of Paperspace, a cloud provider platform, to find out more.
Skills needed for an ML/AI career
Scott Matteson: What type of educational background is helpful in the ML/AI realm?
Dillon Erb: I heard an interesting stat recently: Approximately 70% of deep learning or AI practitioners today are still in school. Because this is an emerging technology, and it’s pulling in people from all sorts of disciplines, we don’t really have a great precedent for it yet.
Truth is, a majority of good practitioners in the space today are either self-taught, or they’re coming from a different domain entirely (i.e. not just computer science or programming). A solid background in statistics and traditional mathematics is always helpful — experience in a research area is also a big plus.
There are also many online courses like FastAI and Udacity — and myriad resources available from all the big tech players — to help educate yourself to become an AI developer. Being proficient on the data side is key and, in particular, Python, as it’s the primary language. Then on the other side, there’s the more traditional software architecture.
Generally today, we have a lot of people who are either really good at the software side and not so good on the mathematical or statistical side. Increasing or balancing your knowledge level and experience in both software and mathematics will give you a significant advantage in the job market.
Scott Matteson: How are traditional IT skills helpful?
Dillon Erb: The challenge IT faces today is that ML/AI is new. It’s a new type of tool that they have to learn about that didn’t exist before. All traditional IT skills are still very helpful and moving forward, there will be more focus on how machine learning tools stack with all the existing systems that are already deployed at an organization. But, right now, there are still questions around how to increase collaboration or visibility within an organization, and how to add more insight to all of the stakeholders. An IT department isn’t necessarily concerned with any single department, but making sure that a decision that’s made in one unit can be either moved to another area if it’s successful or adds a bird’s eye view visibility to the rest of the organization.
This is particularly relevant in the context of data science or machine learning. One of the concerns from IT today is that no best practices exist for AI/ML. The fear is that the practitioners are as siloed as the systems are across different departments, that these practitioners are not using version control for their model building or their software, and they struggle to keep these folks from operating in isolation.
I believe we’ll see a bigger demand for new machine learning tools to play better in a traditional IT context.
Scott Matteson: How are traditional IT skills not relevant?
Dillon Erb: I believe the machine learning universe will ultimately have to conform to a traditional IT process, more so than the other way around. The reason I say that is because IT has, at least in large organizations, broad initiatives like digitization, or collaboration, or very high-level initiatives around increasing developer velocity while still maintaining visibility to outside stakeholders.
Those all will continue to be very strong, but what will have to happen is that the machine learning group, as we discussed earlier, needs to be a hybrid of data scientists and DevOps people. IT will have to accommodate this kind of collaborative unit and try to figure out where it exists in the organization.
There are some areas where AI and machine learning might replace certain aspects of traditional IT today such as threat analysis, anomaly detection, etc. Ultimately, I think it’ll really just be another tool in the toolbox.
Building an ML/AI development team
Scott Matteson: What are the current skills needed to build an ML/AI development team?
Dillon Erb: There are three primary skill sets to look for. The first is generally what falls under data science, or sometimes even BI tooling, which is someone who can gather and clean up existing data, and provide insight into those sources. Then you have an emerging group of AI people who might be less focused on the data collection side and more on building out insights on that data. The third skill set involves a DevOps person who can join teams building out models and prediction engines.
A really killer AI team blends the skills of a statistician or data scientist, and some of the more modern tooling that we generally call AI or deep learning with the DevOps people that can take those models and really push them into production. Today, there’s quite a large gap between those two skill sets.
ML/AI developer job market
Scott Matteson: What does the job market look like for ML/AI developers?
Dillon Erb: It’s still very competitive in the sense that it’s a seller’s market. If you’re proficient in the newer machine learning and AI tools, you will do very well. What’s changed over the last year or two is that back then anyone who could even remotely understand this stuff would have a job. Today, there’s strong pressure towards finding expertise and rewarding that expertise.
Someone coming into the job market and being somewhat familiar with the tools but not able to plug into all the existing systems is far less valuable than someone who can operationalize AI within an organization that already has many systems and many existing tools around data collection/deployment, etc.
Scott Matteson: What new areas of ground are being broken in the field?
Dillon Erb: The most exciting area right now is AutoML. Machine learning models are very hard to create. They require specialists. So the question is, how do you make tools that can automate the discovery of effective neural networks or effective machine learning models?
Another new groundbreaking area is reinforcement learning whereby you create a system in which machine learning models can train themselves, in a sense. I’d also list synthetic data. The idea is almost all machine learning is constrained by how much data is available to the machine learning algorithm. There are exciting new ways of generating new data like using machine learning to generate data, which then trains other machine learning models, which helps bootstrap this whole process.
Scott Matteson: What jobs may be threatened by ML/AI?
Dillon Erb: There’s no question that ML/AI will lead to a shift in jobs in certain areas like document entry, which a machine learning model may do more efficiently and/or be more cost-effective. Data collection in certain areas is another possibility. I do think the technology will create many new jobs as well because as these systems come online, you’ll need people who can monitor them, analyze them, profile them, think about them, and utilize them. Generally, there will be a shift in jobs, but I’m optimistic about the situation in terms of overall job growth.
Scott Matteson: How do you recommend current personnel in any threatened areas evolve to stay competitive?
Dillon Erb: One of the dangers of AI is that in its current form it appears destined to be controlled by a few specialists, or people that have access to extremely large amounts of data. I believe that generally, as a society, or as a culture, we need to invest in understanding these systems better so that they’re no longer black boxes. In fact, they’re the kind of thing that we can talk about collectively — how they perform, and why they exist.
Scott Matteson: What do you foresee down the road for ML/AI and the personnel who work in the field?
Dillon Erb: The big move right now is from R&D into production. Many companies and individuals and researchers have invested in learning this technology over the last few years. The big question now is how do you bring it into a real environment that’s not just the test case.
The longer-term trend is that, and this is a bigger claim, but I believe machine learning and AI will be subsumed into other business practices. In that sense, it’s no longer separate entities, but its actually core in the same way that companies used to have the web team and the mobile team, but ultimately, it became the one app team that did mobile as well as the website.
There’s no question for the people who are looking at it intensely, AI/ML is a fundamentally transformative technology. That said, there are still many open questions about its limits, its bounds. Some of those are technological questions, some are cultural, political, and policy questions. I believe the future of technology is largely undefined, so I would encourage everyone to invest in understanding it better.