Alright. Here I am working through some analytics trending of my Twitter, Facebook, www.adronbhall.com, LinkedIn, and other various sources of traffic. Here's the frustrating part. I don't mean a little frustrating, I am talking about a-grade I almost want to break something frustrating. I have bit.ly, Webtrends Analytics, a database with cross-correlated results, a Twitter monitor in another window, and other little apps to track what is going on. Where does most of this data come together? In my head. Why do I have to manage everything this way, it is damn frustrating!
So here is an idea, and I am just thinking outside the box here (as I always do since nobody will let me in the box anyway :p ). How about these things get merged into a good solid single interface. Maybe even some of the environments get rolled together in addition to the tracking analytics and reports. How about a solid dashboard that allows drilldown into the various mediums to allow actual interaction with the data and platforms? Imagine a dashboard, that reports the analytics for all these things but then lets you just kick in and start tweeting or posting the latest bits to Facebook. Why have a half dozen or more applications and web sites open.
What we need here is some technology leadership.
Others have discussed this recently and are looking for some results. Ok, some of you readers may be thinking, "do you not work for Webtrends Adron? Go build this stuff!" Well maybe others and I am, but I am still frustrated at not having it right now. It seems this is something that is slowly slipping by in this industry and we need a centralizing effort for all these data points to help in actually making sense of them.
A fellow analytics & marketing professional, Rodica Buzescu writes in her blog Morphing Through Time a post titled Web Analytics ? Leaving a Lot to Be Desired. She lays out the current analytics situation very clearly. There is no tool that really bridges the data together easily. Some BI Pros out there may say "oh but I have all that stuff together". This may be true, but how much time and effort did that take? What we need is a centralized management solution for this data. Something to bridge the data together to make it actual, usable, and actionable information.
I have also had a number of conversations with Eric Peterson about various analytics on various platforms. One such paltform is Twitalyzer (@twitalyzer), that he and @katzpdx have put together. This is a prime example however, of a very useful tool, but something that lies outside of other tools. The question is, how to integrate and what to integrate.
As an analytics advocate, practitioner, and developer myself I have one huge issue with a lot of my analytics that I use. Webtrends Analytics & Data Mart help me bring most of those together. However there are a number of things that are still disparate and segmented in the wrong areas.
So with this post as it stands, what do people want joined? Where are all these data points that people want? I think something good just may come out of this desire, but the ideas do have to come together solidly. I have things I can?t disclose, but I know there are minds here at Webtrends thinking about bridging this gap right here and right now.
What I want to know most, though I don’t do much in the way of personal analytics at the moment, is good analysis of where people are coming from. Categorizing referrers, search terms, campaigns, and other sources, because getting found is an important part of it all.
It’s a tough balancing act between a custom-built collection of all your analytics in one place, which can be developed, and building something out of the box that works for most people.
I know that Webtrends has some advantages in that you can use data collection to bring data into the same analytics environment, and data extraction to bring that data into a custom dashboard. Makes me proud 🙂
I would love to see a dashboard that let me populate modules with anything I want!
I think it’s silly to make a big list of what everyone wants to see in their dream analytics dashboard and then struggle with justifying every feature so that everyone is happy. No one will ever be totally happy, because no one solution will fit everyone’s needs. And what if one’s analytics needs change?
I’m dreaming of a dashboard that you can build from scratch or with a template that is based on what a typical business in a certain industry would want as a jumping off point. It would customizable with drag/drop modules that can be filled with whatever data we want, from wherever we want. The data could be connected to other modules, ala Yahoo! Pipes, and exported to whatever form we choose.
Sound good? Now go make it!
<3
Adron, you have virtually spoke the aggrevation of thousands of analysts across the world. I can hear them unite now all chanting "centralized dashboard". But … we have been yelling that for years.
I think 2010 is the year of brining this data together. The hard work of building the systems architecture, scalable databases, algorithms has passed. Heck, we now have jQuery!
That said, while I can’t wait to show you how we have centralized twitter, flickr, youtube and slideshare data into one easy to consume reporting console (www.allinonestats.com) the challenge still remains … of what value do I derive from this coming together. That is something that is being worked out … as analytics users are still trying to understand which web data to view!
I can’t wait to get your feedback on what we have done in New Orleans, and Robin,our vision is to help in that correlation of META data around exisiting pages, users, events 🙂
Oh man, Aaron. Thanks for this post. It’s wonderful to find others who’ve had the same continual frustration as I have. Here’s a long and meandering response.
I’ve been trying to work on a solution to this problem for a very long time. I have some mock-ups, some working prototypes and some sketches. This is something I jumped on immediately after getting into social media work in college. Anthropologists are notorious writers and data collectors. They tend to over-collect data and then sort through it and synthesize it later. Digital anthropologists have the benefit of using or building tools to collect data that comes from lacking the time to have inefficient tools and processes.
In working on the anthropology of social media I can’t just be a "social media person", I must act as one in order to gather my data. I have to step back and look at my work, and that takes time. I have to continuously compress the time and space needed to do my field work, and then do what I can anthropologically on top of that. It makes for extremely long days.
This is why I started talking to people about dashboards and aggregators so early on. I needed one. It’s also why Facebook is so difficult for me and yet so intriguing at the same time. Facebook requires logins in order to receive and post messages, but once one is inside, all of the relevant data shows up to the user. The user only has to float through the data. It’s an addicting and smooth participation architecture.
Bringing data into one place is something Webtrends has talked about at some of the conferences I’ve attended. It seems so simple, which is why it is so difficult The idea of reducing the time and space it takes to access and manage data is basically a holy grail of usability/analytics on the web. It merges the two together. And of course I’d love to say the same thing that you already pointed out before in your post – "You work at Webtrends! Why aren’t you doing this?", I can’t, because I worked at Wieden+Kennedy and I ran into similar problems.
I know how suffocating it feels to have a certain set of ideas and no leadership that cheerleads, gathers resources, or provides a helping hand. I tried to make headway with this at Wieden+Kennedy, but there was limited support. Not to say that I didn’t have it at Wieden+Kennedy. I had tons of help and support internally. My boss and coworkers were fantastic. However, there’s a great difference between having wonderful internal support and communicating that to clients on the outside. Especially traditional clients.
We did some amazingly innovative digital work, for Levi’s and Old Spice and EA Games, but it was expensive work. A lot of the digital work was simply not fundable or understood by the client. Try as we might, with different people, different approaches and different client teams, we couldn’t get through in the way we liked. This sort of thing is a slow process, like trying to get an enormous ship to turn around. One tugboat can do a lot, but multiple tugboats are better. And tugboats are slow, so patience is necessary.
Every once in a while, like on the Coraline campaign, W+K would fork out some ridiculous amount of money (which was possible, legitimate and necessary because they had the funding to do it from the client) for a dashboarding tool and analysis system. This would last for a few months, and then it would be shut off. There was no other choice, but that price-point wasn’t sustainable. The amount of tools and dashboards and media monitoring tools used there in different departments is epic. It has to be that way. Tools change. They are funded by clients. Clients change. Funding amounts from clients change. It’s messy and complicated and just how the advertising industry (and many other industries) work.
It’s hard to come into a company and try to start a new division when there’s variegating funding for it, especially when a series of developers, user interface engineers and designers are needed. This is something that’s difficult to do in most companies, with reason!
I think Waggener Edstrom is one of the only exceptions I’ve seen in this respect. They have an internal team of developers, ect., that create data visualization tools from scratch, and they can use these with clients. Internalizing monitoring tools is a great way to have something unique to offer clients that can’t simply be received by purchasing a system from someone else.
As agencies are forced into the online space, they’re forced to adopt these tools in some way or another. Building an internal reporting system from scratch is an expensive option up-front, but it pays itself off in the end, especially when one considers the learning curve one needs to understand all of the tools that have been used during W+K’s existence. It’s a long term investment with excellent returns for both clients, companies and agencies.
Meanwhile, I’ll continue to use the tools I use. They are getting better with time, but they’re still in adolescence. I’d encourage you to have hope. Frustrations are good for innovation. And I’m sure you understand the history of Webtrends employees. They work there. They learn a lot of analytics. Then they leave and start their own company. Then the company gets acquired by Webtrends. It’s a nice cycle. If there’s one thing I hope to see from you, it’s continued frustration. And of course, if you want to discuss this further, I’m all ears.
Wow Amber! Thx for the comment. Always good thoughts from you.
I’ll keep plugging along, got a few grand ideas, that one day I may have time, effort, and the right project to pull some amazing things together. For now, I know there are some excellent things in the works but it is for a future time to discuss.
I’m really looking forward to a lot of what is in the pipeline for the analytics industry, and of course Webtrends (since I obviously chose to work here!).
🙂 keep reading, digging your insight.
Sadly it is not going to happen. At least any time soon.
Two reasons mainly:
1. It is not possible. : ) Ok perhaps that’s a bit too dramatic. There are too many new sources coming in each day because the world is changing too fast at the moment and we are moving so fast that it is hard to slow down and thing of measurement. For now I think it is ok.
2. I honestly don’t think people with money (big businesses who influence tools by asserting their buying power) care about analytics on the web as much as they care about other things. Money motivates. Once these BB’s get around to caring as much as you and I think things will happen.
Now that does not mean we should not try. I have seen lovely integrations (ok hacks) between twitter, bitly and webtrends. We will keep trying to put data together and try to move the people who want to get moved. But, and you are not saying this, waiting for someone to just build a "global all encompassing dashboard" is imprudent.
Thanks for writing this thoughtful post.
-Avinash.
Adron –
It’s interesting that both bosilytics and mediaChick spoke of finding the value in all this data. There are very few of us who can make effective use of all this data, with our without the tools to get it all in one data mart (integration) or all on the same pane of glass (mashup).
Even before the explosion of social data, there was more data available from web analytics packages than any individual or organization needed. Perhaps counterintuitively, for most people the real challenge is in stripping away the data that isn’t needed to manage the business so that what’s left is truly relevant and actionable. This isn’t to say that the new data isn’t valuable (it can be extremely valuable), but, rather, that all this new data exacerbates the challenge for most organizations.
So, while you are searching for more and more data, many (most?) organizations are struggling to successfully extract the value present in the data they already have.
Switching gears a bit, I think that this is early days on the social data front. This is like the early days of web analytics when we weren’t sure what the core measurements were, and there was no agreement on definitions of things like visits, visitors, sessions, etc. Remember hits? Tools like Twitalyzer and Klout are giving us really interesting data about individual "talkers". They take very different approaches to defining what the measurements are, and which ones are important. They generally don’t agree on what those are, and they’re competing (with others) to win hearts and minds over to their viewpoint. When there is some agreement, I think we’ll see tools like that become sources of data for other analytical applications tracking social.
Good food for thought, Adron. Thanks. 🙂
Hi,
I think this is one of the most interesting post i have seen about he social media because i think only social media makes the internet so interesting and i like it, most of the people still thinks that social media is just a chat but it is not only a chat. There are lots of things to do.
Anyways keep it up and keep continue with your valuable thoughts in the future.
Hi Adron! If this is a discussion about web analysts’ general dissatisfaction with siloed data, I have heard this story before. But I think we have a new twist.
In the early days of web analytics, most of the data we cared about integrating with web site activity was "downstream" from the site itself – eg backend business systems. While "downstream" data is still important, we’re now seeing more and more of an emphasis on "upstream" data – eg the behaviors we call social media.
Social media measurement is a great addition to our arsenal, but then again it complicates things. As Aaron has commented above, analysts have trouble handling the volume of data we currently have at our disposal. As we ask for more, more and yet more data, we need to be certain we can filter the signal from the noise.
Adron,
No disagreement with your thesis or the previous commenters, but I think it’s fun to take the 10K foot view and try and understand why the data cannot be easily integrated. Any guesses? Aaron comes close to the answer when he comments that my Twitter measurement service (Twitalyzer) and Klout take different approaches towards measuring short-message media. The problem is that it’s not just us and Klout — it’s everyone — there are ** no standards anywhere in our sector. **
You guys (Webtrends I mean) have an "open" framework, same as Unica, neither of which look alike in any way other than the same terms are used here and there I suspect. Roll in Coremetrics and Omniture, Google, and Yahoo I suspect we’re up to six different data storage and communication strategies (out of six vendors.) Add to this the multitude of ancillary vendors — the VOC guys, the testing platforms, the performance folks, etc. — and you inevitably have as many approaches towards data as you have vendors. My grandma would have called this a "clustermug", although I don’t think "mug" was what she really meant.
I used to ask why, but now I just shrug and move on.
Ours is an industry where it took nearly six years to agree on the definition of a "page view" and still we have vendors who roll off, drop, and otherwise undercount the damn things behind the scenes. In this context any conversation about data portability and a "universal schema" is more or less DOA. But without any reasonable connective tissue, how could your beautiful dashboard be built?
Sticky-tape, glue, and prayer, which by the way are incredibly expensive in this context and have a half-life measured in months. Sad, but true.
So I applaud you for bringing the complaint back to the table. I love the idea, partially because we have clients who are working on the solution, and partially because it’s a solution that is necessary for this whole sector to move forward. THIS is the technology problem that needs a solution — but people Tweet for hours about how vendor "X" has added the ability to annotate line graphs.
Sad.
I don’t want to tell you what I want joined, I want to be able to join. I don’t want to learn REST, I want to use something like ODBC and MySQL to write "SELECT traffic.page_views, blog.comments, visitor.influence FROM traffic INNER JOIN blog INNER JOIN visitor ON traffic.day = blog.day AND blog.commenter = visitor.name" and get something meaningful to actually visualize. And I sure as heck don’t want to pay for the right to use my data in this way … so at least you guys are getting that right 😉
Neat conversation. Have fun in NoLa.
E.