In a context of COVID-19 pandemic where opportunities for direct interactions have been limited, many of us started to conduct evaluations and provide support remotely. Remote work indeed appears as a practical way to keep operations going. However, it also comes with limitations.
In a first article published in June 2021 – Conducting remote evaluations: what an experience! –, we noticed that experiences could be very diverse according to the context of intervention of SPM professionals. For some of them, working remotely gave the opportunity to provide more tailored, longer term, technical support to FSPs. For others, it turned out to be very challenging and affected the quality of the evaluations. In all cases though, working remotely implied adjustments to the methodology used, and increased effort to compensate for the lack of direct observation.
The experience of Invest in Visions
Invest in Visions, an impact investor based in Germany with whom CERISE is working, has also been conducting social performance evaluations remotely in the past months. One of its Advisors, Enrique Blanco, shares with us some valuable insights on his experience in that matter:
“Due to COVID-19, it was no longer possible for us to perform the assessments in the field. Instead, we decided to conduct assessments on a virtual setting. To replace on-site interviews, we made an intensive use of video calls. However, this approach does not ensure all the appropriate conditions to generate reliable inputs, from which we derive analysis and action plans. For me, the quality of the results is thus affected.
“For instance, when working remotely, it is not possible to be present while a disbursement is taking place. This does not allow to observe with detail the interactions between the financial provider personnel and the client, nor to confirm what are the written documents that are handed out to the client. It is also not possible to assess to what extent the client understood the information provided.
“There are also clear limitations when interviewing the FSP’s field personnel, supervisors and even branch managers. When we conduct interviews in the field, we can make sure that the context is appropriate: in particular, that supervisors are not present or near the interviewed, which could generate bias. When conducting interviews remotely, it is extremely challenging to ensure these proper conditions!
“Working remotely also makes it much more difficult to assess to what extent the organization’s culture is internalized among employees. When we conduct evaluations in the field, we can leverage our experience to notice this when stepping in from the first day of the visit. We can easily identify the gaps between what is officially written and what is observed. When working remotely, these aspects are significantly limited.
“When performing such a thorough assessment for investment purposes, however, you can mitigate risks as described above by performing monitoring afterwards on the field, and periodically. This element, when leveraged properly by the evaluator, puts psychological pressure over the interviewed and decreases the risk of receiving misleading information.”
The experience of Invest in Visions stresses key points of attention to have in mind when conducting remote evaluations. Interestingly, it also shows that, even if remote work can be a pragmatic solution in the current limiting context, it will never be able to fully replace field missions; and that, if on-site missions cannot be performed at an initial stage, it is still essential to plan a field visit at some point, as part of a follow-up or longer-term assistance.
Any tips?
As it is likely that in-person missions will still be limited in the coming months, we asked Enrique Blanco if he had any tip to share. Here is his answer:
- “When working remotely, pay particular attention to those aspects that are hurt the most by the lack of presence on the field: interactions with employees and clients.”
What about you? Would you have any tip to share with the community?
Please send them to us so that we can regularly update this blog with additional inputs!
