Metrics

Discussing metrics on tools such as GitHub Copilot can be a book all of its own. There is a lot of research on how to measure developer output (also referred to as productivity) and what that means. We see customers wanting to take a look at this data as well, but most of them have not started to think of what developer output is, let alone how to compare it.

It is also hard to determine how to measure (developer) output and what a significant improvement in that output would be. Companies often only look at things they can measure, which quickly becomes things such as lines of code, number of pull requests, story points, and so on. However, engineers add value to applications in different ways than just adding extra code – their value is in making the application more performant, more secure, or better able to handle the user’s needs. With the investments that need to be made to enable engineers with GitHub Copilot, companies feel the need to look at developer...