VOK DAMS is a great example of an event agency that has been successfully implementing virtual and hybrid event strategies for the past 18 months. With over 40 years of experience in the field, virtual event spaces aren’t new to them, though the pandemic has accelerated their adoption by clients, bringing to life new and innovative event concepts.
One of the major advantages of virtual and hybrid events is the availability of accurate data that can help with event feedback and evaluation. I interviewed Hannes Putzig, Director of Strategic Solutions at VOK DAMS about measuring the success of hybrid events. Hannes leads the department that is responsible for all things relating to event evaluation—he and his team define the goals and objectives for the event, while the conceptual work and execution are done by a separate department. After each event, Hannes’s department carries out an event evaluation to determine whether it has been successful or not.
This interview took place in December 2020 and is available to watch on YouTube. This blog post has been edited for brevity and clarity.
What are the usual practices and criteria for measuring events success?
We have to differentiate between the usual practices and the best practices. What we see on the market is that the usual practice is to measure 4 to 5 Key Performance Indicators (KPIs) that don’t have anything to do with each other. What is often measured is guest satisfaction, with a quick survey at the end, or just some numbers, such as the total number of participants.
If it is a for-profit event, revenue will often be measured. If it is a corporate event or an internal event, cost-per-participant is often a very important KPI.
Best practice is to make the KPIs work with each other, allowing organisers to see the cause and effect. We try to combine KPIs at certain levels of goal setting—for example, ‘Satisfaction was measured by a certain KPI. And because the satisfaction was high, we had a high turnover or high revenue’.
When we go into a project, we ask the client how they would evaluate the event. The clients usually come up with four or five measurements. We coach the clients on how to set better objectives and how to measure more effectively the achievement of those objectives with KPI scorecards and KPI systems, through which we measure 16 to 20 different KPIs.
We don’t measure them randomly—we connect these KPIs in ways that allow them to relate to each other.
That’s true for all events—that KPIs should relate to each other. Evaluation asks whether success was achieved or not, what the targets were and what the goal was.
We are aiming to diagnose the reasons for success being achieved or not. To answer this question, it is important to not only have quantitative KPIs and an idea of monetary value (how much revenue was generated). It is possible to set out to achieve a certain amount of revenue but the client will never know why they achieved this goal or not if qualitative KPIs were not included.
What is event controlling?
It is a concept whereby, before the event conceptualisation begins, organisers set out to achieve certain objectives. Events should be planned with a purpose in mind. Many clients cannot explain the purpose of an event. They may say they are participating ‘because we always go to the trade show; because this is how we have done it for 20 years’.
It takes a little bit of digging and maybe a strategy workshop to determine the purpose of a client’s event and what are they trying to achieve. It is this goal setting and setting of specific objectives before the event that creates a controlling exercise through which clients identify clear objectives and then monitor and control whether these objectives were achieved or not.
Many organisers carry out their evaluations at the end without thinking about cross-functional control and the end-to-end aspects of event performance measurement.
What is the average time taken for this process and when should organisers start?
It should start before the event’s conceptual development starts.
At VOK DAMS, we begin by defining a clear purpose for the event, with clear objectives that are then broken down into specific smart targets. An example of an objective, at VOK DAMS, we sometimes organise test drives for new cars with the overall aim of raising awareness for those cars. With this objective in mind, we now need to break it down into specific smart goals.
What is a smart measurable goal? For example, 60% of our target group should know of the product’s existence in a post-event survey. This is a specific smart target that I set out before the event and now I can design the event in accordance with it.
Are there any measurement practices that can be regarded as vanity metrics that, in fact, don’t mean anything?
It depends on the event. I cannot say they are clear vanity metrics that people are measuring, but there are a lot of measures that people take for the sake of it, measures that don’t really say anything.
For example, at a for-profit event, the number of participants is very important, especially if tickets are sold. But for a lot of other events, the number of guests in attendance is a very arbitrary metric that is not really important. It doesn’t connect to the overall purpose, and it is not sufficient to measure the ROI of the event itself when organisers planning a marketing event, because they don’t want to have as many people there as possible.
Organisers want to have the right people there; the buyers. I attend many events and many briefings where I hear that ‘we need to reach that many people, we need this size of sample’. It’s important to sample the right audience. Another example is trade shows. When I talk to clients, I ask how they measure the success of their trade shows. And the answer is often ‘the booth was full’. So it was a successful trade show appearance, right? But post-event, we need to look at how many follow-ups occurred, how many business cards were collected, how many leads were really generated. These measures are more important than just the number of guests at the booth.
To sum up, it depends on the event. But, of course, the number of guests is a very, very arbitrary measure, and you should be careful not to treat it as your sole objective.
What are key performance indicators (KPIs) and how do you measure them?
Key performance indicators involve the smart operationalisation of broader objectives. An event has a purpose. Clients come up with maybe 3 to 4 distinct objectives that they try to achieve. KPIs are a way to measure whether an objective has been achieved or not. Clients set out their objectives for the event and they then need to define how to measure them.
I would always set the objectives on several levels. At the basic level, I need to ask ‘Is the right audience (the target group) there?’ I can be very specific and set out two or three objectives for this target audience horizon.
For example, I can say I want C level to be there, or I can say I want at least 200 people for each tour stop of a roadshow. These are clear objectives that are then transferred into a KPI by making it smart and making it quantitative.
A KPI is always something quantitative. It is also possible to quantify qualitative measurements. At the target-group level, it is clear what kind of objectives you can set for yourself. But if we go one step higher, you may ask for the satisfaction of your target audience. Satisfaction is a very qualitative measure; if clients set a KPI for this qualitative dimension, they need to quantify it by asking, for example, how satisfied participants were on a scale from 1 to 5, allowing people to give a quantitative rating. And that is the quantification of a qualitative dimension—it’s the reduction of a complex reality.
Now, especially, with virtual or hybrid events, there’s a lot of engagement on social media whereby people share what they liked about the event or share some quotes from the speakers. Can you measure this ‘social ROI’ and does it count as positive feedback or satisfaction?
Yes. This is definitely a valid option for measuring satisfaction; we can use not only a survey, which is one option, but also a measurement of the social media response. There is a method called media resonance analysis, through which it’s possible not only to analyse numbers but also to set yourself specific targets for the quantity of feedback coming from social media and to measure the sentiment.
It tells you how people are talking about your event online, and it can set specific goals for this, such as ‘We want to have 500 positive reviews on Instagram after the event’. The most important thing is to set the desired targets and carry out the conceptual work once these are set; in this way, it’s possible to make sure targets will be met. We don’t just measure without any clear indication of what the goal is or what the client is trying to achieve.
Many organisers are struggling with low participation numbers for online events— many registrations, low attendance. That means you still capture their data. Is this a good or bad measurement to have?
It depends on the content of the event and the way the organisers have communicated it beforehand. Was it communicated well in advance or late? Is this why there was such a low attendance? The KPI will be used to measure the no-show rate.
No-show rates can be useful, highlighting how many people registered and how many actually showed up. The no-show measurement is a KPI that we very often include in our KPI scorecards because it is important. But for some industries and in some circumstances, a 20% no-show figure is a really good outcome, whereas for others it’s the reverse—80% no-show can be normal. So you really have to be looking at the individual event to assess whether this is a good outcome or not.
Should we compare events with each other, or should each event be measured and evaluated individually?
It comes down to the individual event. If a client takes part in an event series, it’s recommended that they compare the individual events with each other. They need to have a continuous improvement process whereby they come up with the concept for the first event, they measure its success in terms of their smart targets (whether they were achieved or not), and then they modify it for the second iteration and for the third iteration and so on.
So if clients do an event multiple times, it is very, very important to compare the performance of these individual events.
Second, if events have similar goals, they can be compared even if they are not organised by the same client. This is called benchmarking, in which their own events are compared with those of competitors by using any data that can be obtained from competitors. This allows clients to compare events that have the same target and the same purpose.
What I would not advise is for clients to compare events with different purposes and different targets. Don’t, for example, compare the spend on a trade show with the spend on a small-scale activation with the sales team where there’s a seminar. It is really important not to use these arbitrary measures for comparison (for example, ‘I reached this many people there and I reached this many people there’) because if events are compared with other communication measures, events will always lose since they are expensive activities. (Events shouldn’t be compared to newsletter marketing, for example—I had one executive telling me, ‘I reach 10,000 people with my newsletter. Why do I have to do an event for 100 people? I don’t reach that many people with events’.) Clients can reach different audiences with different levels of emotional involvement.
When compared using the right KPIs, the revenue outcome of the event will be different from the revenue outcome, opportunities and bookings resulting from a newsletter. Organisers need to be detailed in terms of what they compare and how they do it.
The events industry hasn’t yet figured out the e-commerce component of virtual and hybrid events. Do you have tips for monetising such events?
There is a lot of experimentation going on right now. Lots of people who did physical events prior to the corona pandemic have now had to shift to virtual and hybrid events with on-site safety measures. But most of it is fully virtual and now they need to monetise these virtual events and that’s a big question I get asked a lot. There’s no one-size-fits-all answer because it totally depends on the event, the content of the event and what amount organisers can ask participants to pay.
I think a good starting point to consider is to charge €100 for an hour in a B2B context. That is typical for a two-hour high-quality content webinar, a two-day virtual conference or a half-hour talk. But it’s not always €100 per hour. There can be dramatic changes from this, both down and up, depending on the content and the target group. What kind of experience is offered? Is it just a normal webinar or a fully immersive virtual reality event?
It is complicated in terms of pricing; what I would say is more interesting than setting a final price tag for your event is how you ask people to pay. Is it just that you buy an entry ticket or are there more intelligent and more innovative ways to ask people for money to attend the event? It’s possible to offer a freemium model or a subscription model where people pay monthly to attend several events.
What we are now seeing is regular up-selling within the event: in-app purchases, like we see with smartphone apps, are really easily incorporated into virtual events. If the entry ticket is sold for the event for a low price, then during the event it’s possible to purchase entry to a workshop for an additional €99, for example, or an exclusive talk with a top CEO for €199. A credit card number has already been provided upon registration, so payment can be made easily. These kinds of in-app/in-event purchases are something that we’re watching closely and are currently experimenting with, and there are plenty of opportunities.
When doing anything for free, there will be a high no-show rate. People don’t feel obliged. Usually, though not always, we recommend charging at least a nominal ticket fee so that people feel that they are obliged; this also encourages organisers to come up with something valuable that people would actually pay for. It is a good practice to also check whether the event programme and content are good because organisers need to ask themselves, ‘Would people be willing to pay for it?’ And organisers should only put out content, run events and deliver experiences that they think people would be willing to pay for. So asking people for just a small amount of money is a good strategy for event organisers, helping to ensure that they are providing valuable content.
No Comments