Developer Skill Maturity Matrix

Going back 25 plus years, the understanding of skill levels of people who work on code around technology projects from small automation scripts, up to building large multi-faceted systems was an elementary basic for leading anything related to coding or development. In the last 5 to 10 years I’ve run into people leading teams, projects, products, etc. that don’t have this basic understanding in start-up up to very large organizations. The lack of this back understanding is just one of the gaps that leads to many of the difficult problems they created for themselves.

But, one of my common sayings is, “You don’t know until you know”. Not everything is known, but if leading a group in a domain, you should know the basics (this skills matrix used to be an essential for getting any software technical managerial role from team manager to top leadership). Sometimes the basics are so common that they aren’t taught, as it was assumed everybody knows the basics. This becomes an issue when the assumption leads to people not knowing.

This is just a relatively simple framing of the matrix.

The Developer Skills Matrix

The developer skills matrix is an underlying maturity model or matrix that is at the heart of any adaptive team framework and organizational maturity tracking. My adaptive team matrix I was using in 2001 was based partly on this developer skills matrix. The skills maturity matrix helps understand where somebody may be and what the next level large step would be with experience and learning. Years of working in one level doesn’t graduate one to the next level, it takes the understanding and capabilities of the next level with experience at that next level. If you are building a team this is essential to know the needs and the fit of people to the needs. This matrix also partially works for helping those who are wanting to move from one level to the next, but there is a lot of learning needed to shift from one to the next and the Matrix doesn’t lay those out. The Matrix is helpful for understanding the knowledge, experience, and differences in mental models (and multiples of mental models used).

The matrix looks at six different maturity levels, but the last (System Designer) doesn’t necessarily have to come from the developer / engineering side, yet it often does (to do it well requires skills and deep understanding outside of the developer / engineering maturity framing). The roles in maturity are:

• Scripter
• Coder
• Developer
• Engineer
• System Engineer
• System Designer

The Understanding and Use of the Matrix

In recent years when I’ve realized this basic foundation of understanding was now missing. It is often management and leaders who haven’t built or run development teams nor teams that ship code that lacked the understanding and need it most. One of the common questions is how long does it take to move someone from one level to the next. The question is one from the wrong perspective, but it often does need answering. The answer varies widely, but roughly 18 months to two years, if the person is willing to learn and can adapt to different mental models. I’ve known people who have gone through the first two quickly (within two years) and are working to become a solid developer quickly, as they have the time and mindset to pick it up. The leap to engineering and getting the engineering mindset doesn’t happen quickly and takes a lot of learning and work. Engineers from one domain to an adjacent domain often takes time as the engineering mindset transfers but the deep understanding of materials takes time and understanding the differences in the systems also takes learning and time to adjust.

The understanding isn’t best from moving people through the Matrix, but what type of skills and person are the fit for the needs of the work. I’ve worked in and run product teams in production and R&D / Innovation Lab environments. Getting the right fit is essential, but also essential for not just the now, but helping people grow. The matrix is also a basic essential for understanding the gaps on a team, or a program. The two toughest jumps are to Engineer and to Systems Engineer.

The difference between an Engineer to Systems Engineer is one is a tough shift and having it become common or natural. It takes the understanding of not only the system being worked on, but thinking of the interoperability and the impacts and concerns on that front as one system interacts with other systems or elements. One of the things that makes this shift to Systems Engineer is the interoperability, which includes learning other sets of materials and systems (and system modeling). The need for a Systems Engineer is one that is often overlooked and one that goes missing in smaller organizations. In cloud services world the Systems Engineer role is one that is deeply needed, but often missing.

The role that was added after the late 90s when I first ran into this Matrix is the System Designer, which most people in that role usually have far more than the light coding capabilities and many times they are or have been engineers with their full set of capabilities and mental models. What separates them is understanding the use and the needs of what is being built from the perspective of the user(s). The users may be other systems and/or people. Understanding the diversity of users and the different mental models and mindsets, as well as expectations of other systems (where interoperability understanding is really helpful) makes for a good Systems Designer. In many organizations an Enterprise Architect (there are many different types and capabilities here as well) is the Systems Designer role if they are focused on the use. The Systems Designers work with the Engineer roles, the product research team(s), and interacting with customers. The Systems Designer is often most helpful if they do have an engineering mindset and capabilities.

From a manager or leadership role, knowing what skills and depth are needed can be difficult. The Matrix helps see and consider the needs of capabilities based on what is being built. In most situations there are fractional role needs above what is considered the core capabilities need. When the Matrix’ existence is known and there isn’t an understanding for skills, capabilities, and/or need reach out for help, as getting it closer to right save pain, delay, rework, and even deep failure.

Using the Matrix in a World with GenAI

The last three years or so with generative AI (GenAI) for coding-related support coming into play I get asked how that changes the Matrix or its use. It doesn’t change the Matrix, but understanding the capabilities and the competence of GenAI models (they are all different) is where the Matrix is helpful. The large language models (LLMs) are trained on everything and they contain good examples and poor examples. They reintroduce poor practices that went away. They also get honed through human training, but the best outputs are largely average and often tilt towards mediocrity. If the use of GenAI is paired with a Coder the results will be less successful than a Developer or Engineer using the similar inputs, planning, and reviews. The difference is more experienced and knowledgeable people that know what should be produced can evaluate and take good next steps.

A GenAI tool can handle scripting relatively well, but that is a large “it depends”. Many Developers and Engineers who don’t use GenAI for coding much often hand off the straightforward script creation and modifications to GenAI to handle quickly and then review them. This is often something handed off to people who were in scripting roles to learn syntax and structure and how to fit in as a Coder in a project or organization. GenAI in knowledgeable and experienced hand can get Coder and Developer level results, but that still needs review and most often modifications and the Coder and Developer can pick that up.

Should an organization replace its Coders and Developers? This is a common question over the last three to four years. Many of the mentoring session discussions I’ve had with people whom are in principle engineering roles and head of product roles, have found that shift to AI for these roles caused a lot of problems with code quality and consistency, and they also have a lot more code to review (a shift and type of the logjam). Most that went down this path more than 18 months ago are working hard to undo the larger problems they face now than what they had prior by leaning hard into GenAI. They are looking to understand how to rehire for roles, but the roles changed to needing people to can sort out large amounts of fragile code that is often not documented or documented incorrectly. The hiring also needs a focus on working to understand the needs to untangle their new more complicated and complex problems, the decisions made as the outcome.

Avoiding the Common Problems with a Different Focus

Those who have avoided the large problems have treated GenAI as augmenting the people they have, not replacing them. They also test and find what works and what doesn’t with regard to the models, and then try to find consistency with what works. Finding consistency can be really difficult with GenAI models, as that is rather counter to how they are built to function. The models drift and knowing that and how to monitor for that shift, or move from GenAI to code that mirrors the good patterns to not only get consistency, but lower AI costs (financial and sustainability costs).

When considering GenAI and fit, the evaluation of where in the Matrix the GenAI model can augment the role and how it can be a fit or not.