In this article, we describe what we in EWDK has been doing concerning evaluation etc. Especially the qualitative analyses for primary target group and stakeholders at a very early stage of EWDK has proved to be very important. Fortunately, we have been free to do the analysis we found relevant, not the ones (as often seen) just carried out in order to persuade donors to continue funding.
This note describes different evaluations that have been used in EWDK. All evaluations serve to ensure the organization and the service that we provide the companies is continuously improved, while other evaluations also serve to evaluate the work that EWDK delivers on certain objectives.
The different types of evaluations can be divided into the following divisions:
- Evaluation of the individual company’s experience with EWDK
- Evaluation of EWDK as a project
- Qualitative analysis aimed at gaining input for the development of EWDK as an organization and development of our services.
- Qualitative analysis deepening our understanding, and how we, in the best possible way, can advise the individual company from the company’s life cycle and the owner’s preferences.
1. Evaluation of the individual company’s experience with EWDK
When a process with a company is finished, the company receives an evaluation form through SurveyMonkey (A translation can be found as Tool 48b). The evaluation form has three focus points; The service from the EW consultant, the service form the voluntary advisor (if relevant) and the service from the lawyer (if relevant). There is a built-in logic so companies are only asked about relevant the service for the individual case.
It is crucial for receiving a satisfying response rate and relevant answers that the evaluation happens at the right time. In Denmark, the closing of a process/case happens in CRM which triggers the evaluation. There are many good reasons to why closing of a case should happen at the right time, but it is especially crucial for the evaluation.
2. Evaluation of EWDK as a project
A traditional evaluation of EWDK has not yet been carried out, where company owners’ satisfaction and the view of various stakeholder groups are examined.
On the other hand, there have been 3 different measurements of effect of EWDK. They have been purely register-based analyses that are not interested in highlighting satisfaction, etc., but focus solely on hard core effects. In Denmark, data is available from the CVR-register (Central Business Register) (Including accounting data for all companies), companies' purchases and sales, company’s exports, number of full-time equivalents etc. In addition, numbers from Skat's (Danish Tax Authority’s) loss in connection with bankruptcy companies has been provided. The effect measurements are generally structured as follows:
- A descriptive part where the EW companies are described in relation to the general company stock and in relation to all companies going bankrupt.
- Identification of a subset of EW companies that have been in a EW course that have been closed for approximately 1-1,5 years before the measurement. This group is highlighted in relation to survivors, survivors' development in turnover, number of man-years, value added and exports. They are divided into whether they have received advice from a volunteer.
- The above subset is compared to corresponding groups from previous effect measurements (survival rate, development in revenue, value added, exports, etc.).
- For the bankrupt part of the abovementioned target group, Skat's (Danish Tax Authority) loss is calculated in relation to a control group.
Implementation of purely register-based effect measurements are of course dependent on access to necessary data in public records.
3. Qualitative analysis aimed at gaining input for the development of EWDK as an organization and improve the quality of our services
In order to gain a foundation for development of EWDK as an organization, we have used qualitative analyses. For primary target group and most important stakeholders specific questionnaires have been developed to highlight significant factors in relation to EW. Each analysis consisted of an individual interview over telephone with an approximately duration of 1 hour with 30 respondents within each target/stakeholder group. The interviews were carried out external, and the respondents are guaranteed full anonymity.
We have carried out the following qualitative analyses in the following order:
- Companies that have been through an EW course: How did they experience it? Was it good? Was it bad? Was there something they missed?
- Our corps of voluntary advisers: What is their motive to working for free? What do they get out of it? What do they think we do for them that they like? And bad? What should be done in order to improve? This analysis was a very important tool for developing and tuning our corps of volunteers.
- Important local stakeholders. In Denmark there are local business-services in most municipalities. Because they are important when dealing with legitimating EWDK, and they play a part in forwarding companies to us, it was relevant to gain overview over their opinion and knowledge about EWDK, our organization, need for information etc.
4. Qualitative analysis targeting a deeper understanding of company owners, and how we, in the best possible way, can advise the individual company from the company’s life cycle and the owner’s preferences.
In 2015 we carried out a qualitative analysis of 9 thoroughly chosen EW-companies. The aim was to make out why some advising works better than others. About the method of the analysis:
- Founded on an interdisciplinary method: Psychological, educational/pedagogical, anthropological, systemically/narrative and professional management approach
- Three perspectives:
The company’s actual life circle
The advisor’s dual positions: Expert position respectively Facility position
The three domains: The many explanations domain, Aesthetics domain and Production domain.
The analysis resulted in several recommendations for our advising and identification of 4 different ‘personas’. Our advising needs to dependent and the advisor needs to know his/her own preferences.
We have created some surveys as supplement to the abovementioned, and based on the knowledge from the qualitative analyses:
- Surveys regarding the volunteers. The aim was to map out competences, effort etc. and gain an overview over their opinion in relation to being part of the corps.
- Survey for approximately 1200 former EW-companies. The aim was not to examine their satisfaction with EWDK, but make out their perception of the cause which lead to the crisis, how it ended and why, how their surroundings reacted, evaluation of their own part in the crisis etc.