Mobile App Exceptions Tracking. Using Alerts in Google Analytics or Google Marketing Platform for Exception Monitoring.

When it comes to mobile apps, retention is a "big deal". It is the most important metric of mobile apps nowadays. Nowadays it is hard to get a customer to download your app, but it is a completely different task to extend a user's Customer Lifetime Value by converting them into a regular and paying user. There are so many engaging, attractive and free apps out there competing for user attention, making the cost of acquiring even a free customer higher than ever before. Consider the number of apps you have on your smartphone today - how many of those are you launching on a daily basis? On average mobile users only open between two and five percent of their mobile apps daily. This means, more than 90% of apps on each person's phone are just sitting there collecting digital dust! From an app developers point of view, these neglected apps are a waste of an install and gaining back this user's attention again will cost the company even more time and money.

So why would a human being go to the trouble of downloading something on their phone, launch it and stop using it? Most people either: forget about the app, just need it once per week, don't have time for it now or had a terrible first experience with the app. Most apps sitting idle on a customer's smartphone have been launched at least once but a lack of engaging content, usability, or need satisfaction means most get forgotten after the initial use.

You can only improve what you can measure.

When it comes to assessing app performance and user experience, Google Analytics is a fantastic tool that can give insight for app improvements. You can find more about assessing app traffic with GA in this post explains how you can implement and utilise Google Analytics to assess your app traffic. The key to customer retention is the customer's first impression and. This includes the app design, content and usability.

No matter what your monetisation model is - the more time user is spending interacting with your app - the more potential you have to make money from it. Do you think a customer will always re-launch an app which has crashed on him?

Each app crash has a massive negative influence on user retention, and as we stated before, retention is most important!

In this article you will learn how to proactively identify and assess app crashes and exceptions so that you can start retaining more of your app users!

Tracking crashes and exceptions in your mobile app

Google Analytics is not a tool for developers - the aim is to help BI & Marketing teams make data-driven business decisions. A consequence of this is that there is little analytical support for developer-focused features such as exception tracking and crash investigation. However there is enough analytical information to gain insight into these features by cross-referencing acquisition data and behavioral metrics.

The following data points are available for each Crash or Exception fact;

  • Has an exception happened? Measurement protocol hit type: exception
  • Whether it is a Crash (fatal) or Exception (non-fatal)? Measurement protocol parameter: exf. For ex.: &exf=1
  • Is there an exception description or any contextual information? Up to 150 bytes, so you can't really fit-in your full stack trace in here. Measurement protocol parameter: exd. For ex.: &exd=My%20method%20has%20failed

Please see Google Analytics Measurement Protocol for further details on each field.

The beauty is that the Google Analytics SDK (Software Developers Kit) will generate these hits automatically and even attempt to deliver fatal exceptions (crashes) when users relaunch the app. Alternatively you can intercept exceptions yourself and use events in the Tag Manager implementation to capture them in GA. Once implemented in your app code you will be able to find connections between retention levels and the number of exceptions in your app.

Proactively monitor app exceptions

Imagine the release of a new app version with lots of new customers features. The marketing team is very excited and increases output (and budget spend) to try and acquire new customers and delivers a message to existing customers that they should update as soon as possible. However, within a week the BI (Business Intelligence) Team identifies that customers are not happy with the new version;

  • Average Session Time has declined more than twice (users spent less time in the app)
  • Some features are not being used and as a result, app monetisation is hurt badly
  • Customer retention and ALV (customer Avg. Lifetime Value) has significantly decreased
  • Despite the fact that a lot of new users were acquired the number of active users have decreased
  • etc.

The above looks as a nightmare, yet it's possible that what is really happening has not been identified: there are a number of crashes or unexpected app performance errors (exception). If you have exception tracking setup, you can cross-reference these findings with the new version and control the damage by rolling back to the previous version.

But it's been a week - could you have known about this sooner? Of course, you would if you go to your Google Analytics daily and proactively check if the number of exceptions or exceptions to sessions ratio has gone up. Luckily, developers don't publish extremely buggy versions of the app store very often (they have their own prevention mechanism for that: Quality Assurance & beta pre-releases) so obviously, you would not check Google Analytics health and stability reports very often. It's best to let Google Analytics notify you if there is a large exceptions increase.

This is where the biggest challenge is, while Google Analytics has a powerful alerting mechanism it can not notify you if exceptions go up.

Please refer to the screenshot below. There is simply no such metric as an exception in the list!

Mobile_App_Exceptions_Tracking._Using_Alerts_in_Google_Analytics_or_Google_Marketing_Platform_for_Exception_Monitoring._image_1

I really hope that the Google Analytics Team will fix this soon but there is always another way! I am going to show how you can still use Custom Alerts in Google Analytics to be proactively notified when exceptions increases.

Using Google Analytics Custom Alerts for MONITORING exception increases

One way would be to deliver exception information to Google Analytics using the event hit type instead of an exception. But this requires changes to your app code and is not a good thing to do if it can be avoided.

A better recommendation is to create a dedicated View in your Google Analytics Property and only allow exceptions to it. Google Analytics Custom Alerts can notify you if there is an increase in users and sessions, essentially this is what the Custom Alerts functionality was built for. So if the view has no data then well done, there are no exceptions, but if the view has a sudden influx of Sessions and Users this means that exceptions are on the rise.

This is how you can do it.

Step 1: Creating the Exceptions Dedicated View

When you send data to Google Analytics - you deliver it to a Property (a GA Property ID usually has the format: UA-XXXXXX-X). GA processing will mirror all data to multiple views. Normally, each GA View in the Property will have a different set of filters applied adjusting incoming data as required.

In your configuration, you might have one View named "All Mobile App Data" receiving all the data (usually no filters applied) or you might have three views: "All Mobile App Data", "Staging or Betas" and "Production". Having a separate view for each app version is possible but not practical.

We are going to create a View which will only have exception data:

  • Navigate to your Google Analytics admin section
  • Select the Property your app is sending data to
  • On the right, click on the Create View blue button
  • Name the View: Exceptions Alerts and configure timezone

Your View will be created and is receiving a full (unfiltered) set of data.

Step 2: Create filter to allow Exceptions only

In the admin section of the new View, pick the Filters link on the left and click on the ADD FILTER red button.

Mobile_App_Exceptions_Tracking._Using_Alerts_in_Google_Analytics_or_Google_Marketing_Platform_for_Exception_Monitoring._image_2

Configure the filter as per the screenshot below. Use Hit Type is exception - Include as the Filter Name, choose Filter Type to be Custom. Next choose Include and select Hit Type as a Filter Field. Enter exception as a Filter Pattern. Click Save.

Mobile_App_Exceptions_Tracking._Using_Alerts_in_Google_Analytics_or_Google_Marketing_Platform_for_Exception_Monitoring._image_3From this moment, your View is only going to accept data if the data is exception or crash (fatal exception).

Optionally, you can go to Google Analytics Real-time reports for this View and see how many users are currently experiencing exceptions. How cool is that!

Because we just have configured the View to filter-in only Exceptions, the View will only have Sessions from Users who has experienced a crash or exception. Therefore an increase in Sessions or Users will signal an increase in exceptions!

What are Sessions and Users are in Google Analytics;

  • Session - is a set of consequent signals sent to GA from a single user within a time frame. By default, it is a 30 mins. In other words, if a user has experienced two crashes within 30 minutes - it will be one session, but if the time-distance between signals is greater than 30 mins - this will be two different sessions
  • User - is a device or an actual user, if he logs in and your app has delivered User ID to GA

As you can see, a number of users in this setup will equal to a number of Users who experience an exception or a crash. Number of session will be less or equal to a number of crashes or exceptions.

Now, let's create Custom Alerts.

Step 3: Creating Custom Alerts to proactively notify you if the number of exceptions has increased

Click on the Custom Alerts button (below the Filters button you clicked in Step 2) and click on the NEW ALERT red button. This is where all the magic is done.

Let's create an Alert for a 10% increase in a number of sessions (almost identical exceptions in the context).

  • Alert Name: 10 % increase in exceptions
  • Make sure it only applies to this View and 0 other views
  • Period = Day
  • Send me an email when this alert triggers: yes
  • (optional) Send a text message when this alert is triggered: yes
  • Alert Conditions
    • This applies to: All Traffic
    • Alert me when Sessions % increases by more than 10% compared to Previous day

Mobile_App_Exceptions_Tracking._Using_Alerts_in_Google_Analytics_or_Google_Marketing_Platform_for_Exception_Monitoring._image_4

Once you click Save Alert, you will be notified on a daily basis if there is a 10% increase in exceptions.

Step 4: Kick back and wait for the alert email to come.

In a similar matter, you can set up the following alerts;

  • Weekly alert if no exceptions (Sessions equal to zero). Everyone loves good news
  • Daily alert if exceptions have increased by 50%. This is a significant increase signalling that may be something went wrong with the new version
  • etc: whatever is required by your DevOps strategy

The above approach will work, but there is the downside of slow reaction speed. Depending on your View timezone you could expect 24-36 hours delay before alert has arrived. This is a limitation of the Custom Alerts and not Google Analytics itself. GA has data regarding exceptions volume within minutes once they happened. To automate this, you can use GA Real-time API or GA Reporting API to grab the data and send an email if you detect an increase.

Please contact me if you require assistance with the implementation or would like to know more details.

If you require any further assistance in the implementation of Mobile App Exceptions tracking, monitoring or anything else analytics related, get in contact with Internetrix today. Internetrix has been a Google Marketing Partner (previously Google Analytics Certified Partner) since 2007 and offer expert knowledge and servicing on Google Analytics products. We are your Australian analytics experts so give us a call or leave a comment with your questions.