Consulting on overcoming the technological
crisis and improving the psychological atmosphere in the company that
develops high-tech software

First of all, the owner of the company had to realistically see the situation on the project and realize the reasons for this state of affairs: no work with the technical debt, no CI/CD implemented, no test management and automated tests and, as a result, no value approach in the team. After several consultations with Dzmitry the customer began to realize the importance of self-organization of the team, the need to transform the values of developers, and approved constant work with the technical debt.

When Dzmitry came to the company and got acquainted with the development team, he saw that there were good specialists working on the project who were making an excellent product. The problem was that they were looking in the wrong direction: they did not focus on strategic planning, they didn’t have an engineering culture, and they weren’t ready to hear each other. Therefore, before proceeding with the technological improvement of the product it was necessary to establish communication processes and relationships between the team and the owner of the company.
Dzmitry initiated constant communication in the team, discussion of the current situation, emerging problems, ideas and prospects. Scrum methodology events started on the project:

1. the practice of daily stand-ups was introduced,
2. demos of new versions of the product were carried out,
3. retrospectives that helped the team to self-organize were conducted.

At this point of time each project participant understood that his opinion would be accepted and taken into account. The owner listened to the team members, and the team listened to his words.
The communication optimization phase took several months. The gradual implementation of Scrum methodology events (retrospectives, stand-ups, demos) made it possible to achieve the first results. According to the customer, in the first three months after the project started, as much new functionality was developed as was previously developed during a year. From the point of view of process management, the situation on the project became more transparent and predictable. The owner began to understand what problems and discontent the development team had and what their causes were. He was also able to make sure that Dzmitry took good care of his business and saw that transformations can be gradual and with low risks, despite the complexity of the situation.

After the implementation of the first stage of improvements in the company and getting positive feedback from the customer, Dzmitry moved on to the implementation of a global plan: work with the technical debts. The project team had the following tasks:

1. Implementation of automated testing (Unit, UI and integration testing);
2. Implementation of Continuous Integration and Continuous Delivery (CI/CD);
3. Legacy code refactoring;
4. Improving product architecture;

Since the system had a complex architecture and an atypical technology stack, the task of CI/CD and test automation introduction was really nontrivial and required research. The product was a desktop solution with installed hardware and software protection, as well as protection through remote servers. When running automated tests directly, it was always necessary to have a USB protection key, so it was necessary to find and apply a suitable UI automated testing framework. The situation was complicated by the fact that the engineers had no previous experience with UI automated testing. The team had a high resistance to writing them: the employees did not have the motivation to work on a difficult, completely new task for them. Therefore, it was decided to start daily research (1 hour per day) on test automation and CI/CD implementation and constantly monitor progress. This approach gradually taught the engineers to work with the technical debts and made it possible to conduct research more efficiently: it was possible to find a solution faster without focusing on one problem. In addition, engineers were now unable to stop working on the technical debt under the pretext of having higher priority tasks.
Significant improvements were made in a few months after starting work on the technical debt. The framework selected according to the results of the research was successfully applied for the purposes of automated UI testing of critical functions of the product. The engineers started writing UI and Unit tests on a regular basis. The Data Driven Testing approach is used in Unit testing. It simplifies input data entry, which is especially important for a system with a large number of calculation modules. Continuous Integration and Continuous Delivery (CI/CD) was implemented despite high team resistance. After 2 months this practice began to bring its first results: the process of creating distributions of the product became fast and safe, which made it possible to significantly free up time and increase the productivity of the team. Continuous work on the technical debt became an integral part of sprint challenges.
One of the important decisions for the systematic improvement of the product was the introduction of architectural hours. The development team held weekly meetings, at which the product development strategy was discussed taking into account its current technical state. Discussion of strategic refactoring was also initiated as part of architectural hours. Based on the results of such events, the CTO drew up an up-to-date plan describing how the code refactoring would be performed in the context of the overall product development strategy.
As part of refactoring, engineers began to work on isolating code modules by covering input and output data with tests. So, during the first year a third of several dozen main modules of the system were isolated in the background. This practice allowed achieving significant progress in the fight against high code coherency and allowed us to minimize the impact of modules on each other.

Professional test management was introduced in parallel with the work on the technical debt. Processes were built and test plans for manual and automated testing were created under the guidance of JazzTeam manager, which made it possible to manage the quality of the product more professionally.
It was also necessary to optimize the processes of collecting requirements, forming and setting tasks. A task processing system (workflow) was created based on the customers’ wishes collected by the team. Further, the process of handling requirements and converting them into tasks was adjusted. The subject area in which the product was applied was particularly complex. Converting requirements into specific tasks was a really laborious process that required additional research and selection of scientific articles. Sometimes there was a need to develop a model, on the basis of which the technical requirements for engineers were created. To implement the strategy, the team was replenished with new business analysts with a scientific background, and time was also allocated for additional scientific research. This approach made it possible to formulate requirements more specifically and accurately. Now the tasks for the development team had the Definition of Done criteria, the engineers were able to independently carry out tasks decomposition and estimation.
A risk assessment system was built. Its essence is in the fact that when specifying the requirements for a specific functionality, a business analyst assesses whether, in principle, a breakdown of critical parts of the product is possible. In this case, he initiates a separate meeting with the entire team, at which possible risks and measures for their treatment are discussed.
Further, the measures to improve the delivery process were taken. Dzmitry introduced the practice of regular sprint planning (separate meetings) and backlog grooming on the project. The team began to conduct a preliminary analysis determining the next release date. If earlier specialists focused on individual tasks due to large uncertainties in development, then, after the introduction of planning practices, they were aimed at implementing the entire set of functionality for the release. Effective planning approaches and implemented engineering practices (CI/CD, test automation) had a positive impact on the product delivery process: the team was able to deliver releases on time. And the number of releases doubled over the year. At the same time, a further increase in release frequency no longer made sense, since customers were not ready to update software more frequently.

As a rule, sudden changes in the processes lead to increased stress levels. Therefore, in order to avoid staff outflow and a decrease in productivity, a special method of introducing changes in the team was applied. The specialists were set clear, adequate, small goals, which they were able to achieve by a certain date, and for some time these goals were scrupulously tracked. A Scrum board was used to monitor progress. On this Scrum board, in addition to tasks, the following charts were presented:

1. being late for stand-ups;
2. readiness for stand-ups;
3. progress in writing Unit and UI tests;
4. progress in CI/CD implementation;
5. daily status of the tests performed overnight;
6. time spent by developers to work with the technical debt;
7. achieving sprint goals.

The purpose of these tools is to capture the smallest steps and improvements so that the team could constantly see progress. Visualization tools clearly showed specialists at what points there are problems on the project, what needs to be worked on and what processes should be constantly monitored. In addition, it helped to monitor compliance with established rules and practices. For example, the chart of time spent for dealing with the technical debts helped establish the practice of writing automated tests by engineers on a regular basis.
Another tool of influence that allowed to increase the culture in the development team is writing documentation by engineers. The meaning of this practice is to improve the quality of the work performed by specialists. When the engineer realized that he had to write a clear manual for using the created utility, he approached the development process more thoughtfully. It was supposed to be a complete functionality, the work of which can be depicted with a simple diagram and a clear sequence of actions.
To increase the effectiveness of various meetings and discussions, each engineer had a workbook in which he had to mark the tasks assigned to him, so that he could then independently write them on the Scrum board. Thus, all the discussions were not wasted, the tasks and questions received were not forgotten and were always processed. In addition, it made it possible to track the context of discussions in controversial situations.
Every week, the developers demonstrated the results of the work done during the week, answered questions, received wishes and comments from the team (team demo). After such an event, each project participant understood how certain parts of the product function. On the other hand, each engineer could share the successful solutions applied in the process of working on the task.
With the first successes the project participants gradually gained confidence that they would be able to get out of the difficult situation on the project. This helped to overcome learned helplessness. Engineers began to look for ways to solve problems, and not the reasons why these problems could not be solved, finally saw the results of their work and realized that they had gotten off the ground. The atmosphere in the team became more friendly: communication improved, the owner was now on the same side with the team. The motivation to accumulate the best practices increased and an understanding appeared that the efforts of each person really make the product better.