Consulting on overcoming the technological
crisis and improving the psychological atmosphere in the company that
develops high-tech software

Project implementation period: 1.5 years
Other cases

Business challenge

Initially, the product developed as a startup successfully entered the market and took its share. It had no analogues in the world, so the market actually formed from scratch. When the process of business development reached a certain milestone, the owner of the company decided to conquer new market niches, which required the addition of a significant part of new functionality. However, by this time, the development team had reached a deadlock and for several years was engaged in product support only. To be able to continue developing the product, it was necessary to transform the culture of the team: to transfer engineers from the support mode to the creation of new IT solutions. However, this goal could not be achieved independently, since from the technical, process and psychological sides the project came to a point where further business development was impossible.

Collaboration context

At the beginning of the project life cycle, the company owner had an informal conversation with Dzmitry on technical nuances and further development of the product. Initially, in 2013, Dzmitry shared his point of view on the need to apply the Data Driven Testing approach. He also advised to organize the delivery process via CI/CD. However, at that moment the customer preferred the company’s rapid growth rather than the gradual implementation of engineering practices bringing value in the long term. The development team focused on the rapid expansion of functionality and the release of the product to the market. The chosen approach had an effect: the company really grew rather quickly, but after 4 years the product owner saw problems with stability, delivery and non-scalability of the system. Then he remembered Dzmitry’s recommendations and in 2019 decided to turn to him for help.

Situation on the project

By the time the owner of the company turned to Dzmitry for consultation, the project had already achieved significant success. A small startup has grown into a full-fledged successful business showing steady sales to clients. However, it was difficult to continue development: from a technical point of view, it became difficult to modernize the product, product releases were extremely unstable and deliveries became irregular, there were serious problems with the introduction of new employees into the team, the BUS factor (critical dependence on specific employees) increased. In addition, the team had problems with culture, communication and organization of processes. The following factors led to this situation on the project:

  1. Product technological complexities. Initially, the project was developed as a startup, so many important practices and approaches were not applied. Accordingly, the principles of Continuous Integration and Continuous Delivery (CI/CD) were not implemented. Writing Unit tests was a rare practice applied ad-hoc: the code coverage was about 10%, and in case of any code changes, a large number of bugs arose. There was also no automated UI testing, since its implementation was an even more complex and labor-intensive task that required the use of a separate framework. Manual testing was carried out on an ad-hoc basis, no regression testing was carried out. Test management was not organized, as a result of which the necessary quality assurance processes were not built.
    The system became very complex due to the accumulation of a large amount of source code of different versions written by different generations of employees. Engineers working on the product used different approaches to the culture of coding, the principle of continuity in architecture was not respected.
    So, the product reached a critical amount of technical debts and could not be further developed. Strong linkages between technical debt issues prevented the team from working on one of them without affecting others. Before implementing the Continuous Integration and Continuous Delivery process, it was necessary to refactor the system. However, it was impossible to objectively assess the success of refactoring, since there were no Unit tests, so attempts to refactor were painful and difficult. Without refactoring skills and experience, the team did not develop strategic thinking, as a result of which engineers could not work systematically to improve the architecture. In addition, it was difficult to develop the architecture without automated control over stability: any changes could lead to new bugs. In such a situation, even product support was very expensive and required significant labor costs.
  2. Processes were not established. The project lacked a business analysis process. Requirements from product customers were collected on an irregular basis. The team received requirements in the form of wishes, and they entered them into the tracking system (Jira) in the same form, but already as tasks for developers. So, the tasks had no performance criteria and no description of how the desired functionality should look and work.
    The practice of setting tasks for iteration wasn’t established in the project. The tasks were worked over unsystematically, new requirements were constantly added, priorities changed all the time. Deadlines were determined based on the assumptions of the development team.
    Lack of planning and systematic work on tasks combined with a large number of bugs caused problems with deliveries in case of any changes. As a result, releases were carried out once a year with delays of up to 4 months.
  3. Communication problems, high toxicity in the team. From a psychological point of view, the business owner and the developers became out of sync and could not understand each other. The team did not have an engineer who could draw the owner’s attention to the need to work with the accumulated technical debts. And the owner himself did not yet consider it necessary to invest in such work. Constant dissatisfaction of the engineers and negative feedback from the owner led to the fact that the mood in the team turned spikiest and the level of toxicity increased.
  4. Learned helplessness. The engineers did not see an opportunity to continue developing the product in such conditions. Faced with new difficulties in the process of changing and expanding the code, they gave up and did not even try to overcome new obstacles.
Main problems on the project
  1. Low productivity of the development team due to the huge volume (more than 400,000 lines) of highly linked legacy code that cannot be changed.
  2. Lack of manual and automated testing and, as a result, regular delivery failures and unpredictable bugs in the product.
  3. Poorly established processes (development planning, business analysis, testing, software delivery, administrative processes).
  4. Toxic atmosphere among teammates / misunderstanding between the product owner and the team.
  5. Learned helplessness among the engineers.
Results of comprehensive work
from 8% to 22%
from 8% to 22%
the code coverage by Unit-test increased
3 times faster
3 times faster
new employees immerse in the project
At twice
At twice
new versions of the product release accelerated and the amount of functionality implemented for the release increased
by more than <br>15 times
by more than
15 times
the time spent by a specialist for the build of a license increased

Feedback on the results of comprehensive work

Consulting format

By agreement with the owner, the consultation took place in the following format: Dzmitry began consulting with personal communication with the owner and team members, audit of the situation on the project. When the main direction for improvements was outlined, specific on-site assistance (monitoring the implementation of improvements, mentoring the team) was provided by JazzTeam project manager. At the same time, Dzmitry regularly joined the team, conducted an analysis and built a further strategy relying on the actual state of things on the project.

Consultation roadmap
  1. Dzmitry advises the owner of the company in a personal communication format.
  2. Dzmitry and the project manager immerse themselves in the customer’s company, analyze the situation, draw up a vision of the main processes and agree on it with the CEO and СТО of the customer company.
  3. Dzmitry and the project manager carry out the first two iterations of changes, and a general retrospective is held based on the results of each of them. The results are analyzed.
  4. The manager remains on the project, and he continues to implement all the processes. Every two months Dzmitry joins the project, evaluates the results and develops a further improvement strategy.
  5. For the customer the final demo of the improvements made is held, the results of cooperation are summed up.

Approaches and solutions

Knowing about Dzmitry’s reputation and his successful experience in developing his own company, the product owner fully trusted him. Accordingly, the management gave the green light to any changes, which positively influenced the success and high speed of achieving the results of consultation.

Work directions
  1. Establishing transparency of all communications on the project
  2. Working with the technical debts (including setting up technical processes: CI/CD, Unit and UI testing, architectural planning)
  3. Setting up project management, testing and business analysis processes
  4. Working with the culture and values of the team

Work stages

  1. Stage 1:
    Personal consultation to the company owner

    First of all, the owner of the company had to realistically see the situation on the project and realize the reasons for this state of affairs: no work with the technical debt, no CI/CD implemented, no test management and automated tests and, as a result, no value approach in the team. After several consultations with Dzmitry the customer began to realize the importance of self-organization of the team, the need to transform the values of developers, and approved constant work with the technical debt.

  2. Stage 2:
    Communication

    When Dzmitry came to the company and got acquainted with the development team, he saw that there were good specialists working on the project who were making an excellent product. The problem was that they were looking in the wrong direction: they did not focus on strategic planning, they didn’t have an engineering culture, and they weren’t ready to hear each other. Therefore, before proceeding with the technological improvement of the product it was necessary to establish communication processes and relationships between the team and the owner of the company.
    Dzmitry initiated constant communication in the team, discussion of the current situation, emerging problems, ideas and prospects. Scrum methodology events started on the project:

    1. the practice of daily stand-ups was introduced,
    2. demos of new versions of the product were carried out,
    3. retrospectives that helped the team to self-organize were conducted.

    At this point of time each project participant understood that his opinion would be accepted and taken into account. The owner listened to the team members, and the team listened to his words.
    The communication optimization phase took several months. The gradual implementation of Scrum methodology events (retrospectives, stand-ups, demos) made it possible to achieve the first results. According to the customer, in the first three months after the project started, as much new functionality was developed as was previously developed during a year. From the point of view of process management, the situation on the project became more transparent and predictable. The owner began to understand what problems and discontent the development team had and what their causes were. He was also able to make sure that Dzmitry took good care of his business and saw that transformations can be gradual and with low risks, despite the complexity of the situation.

  3. Stage 3:
    Technical debts

    After the implementation of the first stage of improvements in the company and getting positive feedback from the customer, Dzmitry moved on to the implementation of a global plan: work with the technical debts. The project team had the following tasks:

    1. Implementation of automated testing (Unit, UI and integration testing);
    2. Implementation of Continuous Integration and Continuous Delivery (CI/CD);
    3. Legacy code refactoring;
    4. Improving product architecture;

    Since the system had a complex architecture and an atypical technology stack, the task of CI/CD and test automation introduction was really nontrivial and required research. The product was a desktop solution with installed hardware and software protection, as well as protection through remote servers. When running automated tests directly, it was always necessary to have a USB protection key, so it was necessary to find and apply a suitable UI automated testing framework. The situation was complicated by the fact that the engineers had no previous experience with UI automated testing. The team had a high resistance to writing them: the employees did not have the motivation to work on a difficult, completely new task for them. Therefore, it was decided to start daily research (1 hour per day) on test automation and CI/CD implementation and constantly monitor progress. This approach gradually taught the engineers to work with the technical debts and made it possible to conduct research more efficiently: it was possible to find a solution faster without focusing on one problem. In addition, engineers were now unable to stop working on the technical debt under the pretext of having higher priority tasks.
    Significant improvements were made in a few months after starting work on the technical debt. The framework selected according to the results of the research was successfully applied for the purposes of automated UI testing of critical functions of the product. The engineers started writing UI and Unit tests on a regular basis. The Data Driven Testing approach is used in Unit testing. It simplifies input data entry, which is especially important for a system with a large number of calculation modules. Continuous Integration and Continuous Delivery (CI/CD) was implemented despite high team resistance. After 2 months this practice began to bring its first results: the process of creating distributions of the product became fast and safe, which made it possible to significantly free up time and increase the productivity of the team. Continuous work on the technical debt became an integral part of sprint challenges.
    One of the important decisions for the systematic improvement of the product was the introduction of architectural hours. The development team held weekly meetings, at which the product development strategy was discussed taking into account its current technical state. Discussion of strategic refactoring was also initiated as part of architectural hours. Based on the results of such events, the CTO drew up an up-to-date plan describing how the code refactoring would be performed in the context of the overall product development strategy.
    As part of refactoring, engineers began to work on isolating code modules by covering input and output data with tests. So, during the first year a third of several dozen main modules of the system were isolated in the background. This practice allowed achieving significant progress in the fight against high code coherency and allowed us to minimize the impact of modules on each other.

  4. Stage 4:
    Setting up the processes

    Professional test management was introduced in parallel with the work on the technical debt. Processes were built and test plans for manual and automated testing were created under the guidance of JazzTeam manager, which made it possible to manage the quality of the product more professionally.
    It was also necessary to optimize the processes of collecting requirements, forming and setting tasks. A task processing system (workflow) was created based on the customers’ wishes collected by the team. Further, the process of handling requirements and converting them into tasks was adjusted. The subject area in which the product was applied was particularly complex. Converting requirements into specific tasks was a really laborious process that required additional research and selection of scientific articles. Sometimes there was a need to develop a model, on the basis of which the technical requirements for engineers were created. To implement the strategy, the team was replenished with new business analysts with a scientific background, and time was also allocated for additional scientific research. This approach made it possible to formulate requirements more specifically and accurately. Now the tasks for the development team had the Definition of Done criteria, the engineers were able to independently carry out tasks decomposition and estimation.
    A risk assessment system was built. Its essence is in the fact that when specifying the requirements for a specific functionality, a business analyst assesses whether, in principle, a breakdown of critical parts of the product is possible. In this case, he initiates a separate meeting with the entire team, at which possible risks and measures for their treatment are discussed.
    Further, the measures to improve the delivery process were taken. Dzmitry introduced the practice of regular sprint planning (separate meetings) and backlog grooming on the project. The team began to conduct a preliminary analysis determining the next release date. If earlier specialists focused on individual tasks due to large uncertainties in development, then, after the introduction of planning practices, they were aimed at implementing the entire set of functionality for the release. Effective planning approaches and implemented engineering practices (CI/CD, test automation) had a positive impact on the product delivery process: the team was able to deliver releases on time. And the number of releases doubled over the year. At the same time, a further increase in release frequency no longer made sense, since customers were not ready to update software more frequently.

  5. Stage 5:
    Changing the culture and overcoming the team's learned helplessness

    As a rule, sudden changes in the processes lead to increased stress levels. Therefore, in order to avoid staff outflow and a decrease in productivity, a special method of introducing changes in the team was applied. The specialists were set clear, adequate, small goals, which they were able to achieve by a certain date, and for some time these goals were scrupulously tracked. A Scrum board was used to monitor progress. On this Scrum board, in addition to tasks, the following charts were presented:

    1. being late for stand-ups;
    2. readiness for stand-ups;
    3. progress in writing Unit and UI tests;
    4. progress in CI/CD implementation;
    5. daily status of the tests performed overnight;
    6. time spent by developers to work with the technical debt;
    7. achieving sprint goals.

    The purpose of these tools is to capture the smallest steps and improvements so that the team could constantly see progress. Visualization tools clearly showed specialists at what points there are problems on the project, what needs to be worked on and what processes should be constantly monitored. In addition, it helped to monitor compliance with established rules and practices. For example, the chart of time spent for dealing with the technical debts helped establish the practice of writing automated tests by engineers on a regular basis.
    Another tool of influence that allowed to increase the culture in the development team is writing documentation by engineers. The meaning of this practice is to improve the quality of the work performed by specialists. When the engineer realized that he had to write a clear manual for using the created utility, he approached the development process more thoughtfully. It was supposed to be a complete functionality, the work of which can be depicted with a simple diagram and a clear sequence of actions.
    To increase the effectiveness of various meetings and discussions, each engineer had a workbook in which he had to mark the tasks assigned to him, so that he could then independently write them on the Scrum board. Thus, all the discussions were not wasted, the tasks and questions received were not forgotten and were always processed. In addition, it made it possible to track the context of discussions in controversial situations.
    Every week, the developers demonstrated the results of the work done during the week, answered questions, received wishes and comments from the team (team demo). After such an event, each project participant understood how certain parts of the product function. On the other hand, each engineer could share the successful solutions applied in the process of working on the task.
    With the first successes the project participants gradually gained confidence that they would be able to get out of the difficult situation on the project. This helped to overcome learned helplessness. Engineers began to look for ways to solve problems, and not the reasons why these problems could not be solved, finally saw the results of their work and realized that they had gotten off the ground. The atmosphere in the team became more friendly: communication improved, the owner was now on the same side with the team. The motivation to accumulate the best practices increased and an understanding appeared that the efforts of each person really make the product better.

Consulting strategy

Working on this project, Dzmitry applied all his extensive experience: he used the skills of an architect, manager and CEO of his own IT company. This triad made it possible to effectively transform the culture of the customer’s company.

Technological side and product architecture

Dzmitry communicated directly with the engineers of the team. Thanks to his technical background, he managed to quickly gain credibility, prove the importance and effectiveness of the best development practices. As a result, the team began to approach the process of working on the product more meaningfully, actively discussing the events taking place on the project. For example, engineers began to identify reasons for failing tests or debate why a new build of the product was unsuccessful. This indicates a change in the values of specialists and the establishment of the engineering culture in the team.

Setting up development processes

Being guided by his experience as the project coordinator, Dzmitry built processes in the customer’s team (testing, business analysis and delivery) and also introduced the practice of planning various flows, the culture of decomposition and estimation of tasks.

Managing administrative processes and establishing the culture in the company

Dzmitry passed on the best practices and management techniques, on the basis of which he built processes in his own company. As a result, the customer company changed the approach to engineering and organization of processes at the value and cultural levels.

Having started changes with the optimization of communication and the technological core of the product (with focus on CI/CD and Unit tests), Dzmitry managed to spread the influence of the value culture on almost all aspects of the company’s life: business analysis, administrative management and business development. So, the customer received more than just technological or process consulting. Ultimately, the improvements affected all aspects of the company’s life.
Thanks to the client’s complete trust in Dzmitry, a high speed of improvements implementation was achieved. In 1.5 years the company with imperfect processes and non-transparent management was transformed into the engineering company with the modern culture and values, which became a springboard for further product scaling and growth. The system received a new impetus for development. At the same time, all the key members of the team were retained, despite the scale of the changes and the colossal changes in the daily work and outlook that each employee went through.

Customer's achievement as a result of cooperation

  1. The owner of the company realized the need to constantly work with the technical debt, came to the decision to invest in CI/CD and became a constant driver for creating new automated UI tests. Over a year and a half the code coverage with Unit tests increased from 8% to 22%.
  2. He works to eliminate the technical debts was established:
    • The customer’s team implemented the practice of writing automated tests (Unit, DDT, UI) on a regular basis, which made it possible to make changes to the product safely and efficiently.
    • The CI/CD culture was introduced, thanks to which the product delivery process became significantly faster and more stable.
    • Systematic work to refactor and improve the architecture of the complex system was established.
  3. The following processes were organized:
    • Business analysis/creation of requirements to software.
    • Risk assessment.
    • Development planning.
    • Quality assurance.
    • Software deliveries.
  4. A value-based approach to the development process was formed in the team. Prior to our joining the project, the main goal of the team was the rapid implementation of the functionality – the specialists ignored product quality and support. Three months after the start of the cooperation the customer’s development team began to reduce the technical debt, use and accumulate best engineering practices, maintain the necessary documentation, while increasing the speed of creating new functionality.
  5. The psychological climate in the team was improved, the motivation to overcome learned helplessness increased. Toxicity was significantly reduced, the healthy working environment was established on the project, and communication and relationships between the company owner and the development team were established.
  6. The Scrum methodology was introduced, that accelerated the development process and made it understandable, transparent and manageable for the customer.
  7. The budget was saved by making product build cheaper. Automatic build greatly stabilized, simplified and accelerated the process of creating different product versions for delivery to customers. The time spent by a specialist for the build of a license increased by more than 15 times (10 minutes versus ~ 3 hours). This freed up additional time for development, and also removed the dependence on previously irreplaceable team members.
  8. It became possible to painlessly add new developers to the team. The number of developers on the project doubled.
  9. Thanks to the comprehensive work, the customer was able to release new versions of the product 2 times faster, while the amount of functionality implemented for the release also doubled.

Review

Dmitry Evlanov's review of consulting

“Dzmitry Harachka is able to take into account the interests of everyone. The project is very difficult, with a large amount of technical and organizational debt. And the fact that it was possible to accomplish a serious organizational transformation without rattling a saber, and at the same time to retain talents and not allow demotivation – that’s no small feat.
I especially liked that Dzmitry did not just give specific advice, but directly managed the process of cultural development transformation. An extremely rare combination of technical skills, empathy and mediation skills. Not everyone knows how to be a leader for both management and engineers.”

 Dmitry Evlanov's review of consulting
Dmitry Evlanov
СЕО и founder of Simmakers.com
Contents
  1. Business process
  2. Main problems on the project
  3. Results of comprehensive work
  4. Consulting format
  5. Approaches and solutions
  6. Work stages
  7. Consulting strategy
  8. Customer's achievement
  9. Review
rect
Get FREE checklist «How do you know if your project is in danger?»
photo

Greetings! My name is Dzmitry Harachka. Specially for you, I have prepared a checklist that will help you determine if your project is in danger and provide a clear quantitative assessment of the complexity of the situation.

In the process of creating this document, I applied many years of my experience in managing, coordinating, supervising and consulting on more than 100 IT projects. Every checklist item is a reflection of the most common challenges I have seen in the development teams during my consulting practice.

Moreover, the checklist contains a large number of practical tips: this will help you move forward in solving problems on your own.