Common Analytics Assumptions — Whiteboard Friday

Discover common analytics assumptions with Dana in this Whiteboard Friday. A must-watch for marketers seeking clarity in the ever-evolving landscape of analytics.

Howdy, Moz fans. I'm Dana DiTomaso, I'm president and partner at Kick Point, and I'm also the founder and lead instructor of KP Playbook, which is a new marketing training site we've launched.

What I'm gonna be talking to you today about is common analytics assumptions. And I'm sure that some of you have used analytics for a very long time. Some of you may not have used analytics for very long at all. You've all heard about GA4, and you've thought, what is this garbage? Why do I have to deal with it?

Well, I mean, as my button says, I Heart GA4, I do. I promise it is a good product. It's just the user interface, eh? But one of the things that I find the most difficult with GA4 is it exposes the issues that we've had over time where analytics has never actually been all that accurate. Just now, we know it's not that accurate.

So if you've been presenting data to, say, clients or your boss as absolutes and now you realize it isn't actually true, you might be struggling with some of these assumptions, which is what I'm gonna walk you through today.

So the first assumption we're gonna talk about is I can track everyone. Maybe at some point in ye olden internet days, you could actually track everyone, but now people have ad blockers. I use one, you might use one too.

The problem is that ad blockers stop people from, say, running your analytics scripts. That's what happens. The other thing to think about is intelligent tracking prevention which is a tool used by Safari and Firefox browsers. It removes cookies after seven days. So if I come on your website on day one and I come back again on day eight, I am a brand new person as far as my analytics is concerned. What are you gonna do about that? Literally nothing. That is gonna get reduced soon to 24 hours. And then you're really not gonna know if people are coming back or not.

Then the third thing, humans are weird, right? Like sometimes, if I see a really great ad and I'm gonna buy that thing, I'll do it right away in that same session with my ad blocker turned off to reward that paid marketer for doing a great job with that ad. Now, you're not always doing that. Sometimes you might send a link to say your spouse and say, hey, can we buy this? And then they click the link in the message, and then they buy it through there, and it shows up as direct, and you have no idea what actually happened. People use the internet in weird ways. Watch your family use the internet, watch your friends use the internet. You will see the strange things that they do because they approach the internet, not like a marketer, unlike the rest of us.

So really keep that in mind, and especially if you're having trouble explaining things to your clients about this, ask them to use the internet, show you what they do, and you'll really start to see where these issues can crop up and how people use internet in weird, and odd ways. Like my father-in-law, bless his heart, he used to Google Google, and then he would Google what he was gonna Google. Like those kinds of people still use the internet.

Next thing we're gonna talk about is this is how many users we had. People will present this as an absolute number. This is the number of users. In Universal Analytics, you probably had the number of unique users, number of new users, number of returning users.

Again, if we think back to what I just talked about with intelligent tracking prevention, everything else, that was kind of a lie probably. So first thing, you don't actually get to track users unless you have logins on your website. For example, the site that you're on right now watching this video, Moz, they have logins. They can track you if you come back if you're logged in. If you never logged in, they probably don't know who you are if you're using Safari and you're coming back eight days later. But if you are logged in, we can track you across multiple sessions.

Most websites do not have logins. Many business-to-business websites don't have logins. SaaS products do, good for them, but most of us don't have the luxury of logins. And so that means that we don't actually know who a user is for some of the reasons that I just talked about. But also other stuff, like people use different computers, right? Like people come back years after they first came back. We don't really know how many users you have.

And also, have you actually tracked users because some people might say, oh, we have user logins, but unless you've informed GA4 of who these users are, unless you've gone through the configuration steps to actually send that user data off to GA4, Google doesn't actually know who users are. Just having the login isn't enough. You need to set up extra stuff in order to get those users reporting. To do that, I would recommend searching for GA4 user reporting, and there's step-by-step instructions there from Google on how to set that up.

Okay, third point, bounce rate. No, bounce rate is gone. I mean, technically, there's bounce rate in GA4. We're gonna talk about that a little bit in a second. Engagement rate is showing how people use our website. I mean, is it? I feel like that that GIF right now with Thor saying, "Is it?"

The thing is, with engagement rate and bounce rate, what is it actually measuring? In GA4, engagement rate is measuring the number of people who had your website as the active tab for at least 10 seconds, or you can change that, the default is 10 seconds. You can go up to 60 seconds of the admin settings or people who visited two pages or someone who converted, or someone's first visit to the website. That means that they are a user. That means that they had an engagement on your site.

Other people, for example, someone who has been like hoarding a tab for six months, those people aren't gonna show up as an engaged visit if they come back to the website because maybe they only open the tab for one second, for example. Or maybe you up that limit to 60 seconds and someone was on the website for 59 seconds, not engaged anymore. So you really need to think about what that means.

And bounce rate in GA4 is not the same as bounce rate in Universal Analytics. That is a different video entirely, but I'm gonna say it's not the same. While there is a bounce rate in GA4, alts measuring is the inverse of engagement rate. So really, just report an engagement rate. Bounce rate never meant what you thought it meant. It was a garbage metric anyway. Let's all remove ourselves of it and get it out of the world and let's focus on engagement rate instead.

And of course, last but not least, you probably saw this little bonus that I have down here. GA4 not set. This is a huge problem that people are seeing in GA4. They're like, "I looked at my landing pages, and a bunch of them were not set, or I was looking at my acquisition, and a bunch of it was not set. What is going on?"

So this is how GA4 works and how it's different from Universal Analytics. GA4 is measuring a series of events that happen on your website, a page view, a scroll, someone clicking on a call to action button if you've set it up to track that. If I have, say, a tab open on the website, maybe this tab you're on right now, maybe you clicked it from, say, a newsletter, and then you decided I'm gonna watch that video later. And then you came back later, and you clicked play in the video, that page didn't reload. There's no more page view event. But because you clicked play on this video you're watching right now, there's a video play event.

The problem is that because there's no page view, we don't know how you landed on the website because that page view event, that session start event, goes along with that initial page view. So if you had come back to this tab, hit reload, and then hit play, we would know where you came from. But because you didn't, you just hit play, maybe go back and reload, and then hit play, so we know where you came from; the problem is it's gonna show up as not set.

So if you're seeing a lot of not set in your GA4, there's a little tiny part of it that is a bug that Google may or may not fix. But the other part of it is that people are hoarding tabs. How many tabs do you have open right now? I bet it's more than this one. And so because of that, you're definitely gonna see a lot of not set in your reports. The amount of not set that you see will depend on the number of people who hoard your tabs.

Hoarding tabs isn't necessarily a bad thing. People like to read stuff later. People might be like, I'm gonna think about this product for a week and then buy it. So this can tell you how many people are doing those things. But it is something to think about when you're doing that reporting. Should you include that not set data? Maybe report on it in a slightly different way.

So if you're seeing not set, don't stress about it. That's what's going on. You may or may not wish to include it in your analytics. I find that when you're reporting to people who are not doing what we do, such as leadership at your company or your clients, it's sometimes best to get rid of it 'cause it just confuses people who aren't us. But certainly, you know, depending upon the kind of data that you're capturing with not set, you might wanna report on it or not report on it.

And so with that, those are common analytics assumptions I find that people are making that are mostly incorrect. I hope this helped you in your analytics journey and I promise that eventually, over time, we will all love GA4 because we're kind of stuck with it now. Thanks for watching.