I know it's highly-coupled to your particular infrastructure, but would it be possible to open source?
I don't think there's much to open source -- it's about 2,000 lines of code. It's mostly just writing custom sql and elasticsearch queries to show data related to the users we're about to respond to.
Re: the log integration, all our application logs get written to logstash/elasticsearch/kibana, and one of the fields on the logs is the person_id (they're unique id whether logged in or out). When we show the logs in the feedback center, we just query elastic search and group by the different log types.
Seems like there would be a lot of overhead for that amount of logging though. Could you elaborate a bit more on how it all works? Do you run an internally developed mixpanel/GA type tool? Or, is this server side logging (the JS issue speaks otherwise).
For the JS logging you see in some of the screenshots, we use some JS on the client side that watches for errors and fires off a request to us with as much data as on the error as we can collect (in most cases we can get stack traces, which is pretty nice).
In terms of where this all gets stored and aggregated, all of our application level logs are shipped into Elasticsearch, and that's where the aggregations and counts you see in the screenshots come from. All of the application level logs are sent in with a unique identifier per-client so that we can keep track of them that way, as well as tagged by username, IP address, and anything else that might be useful in cross-correlating with logs from the rest of our stack.
If there's any more specific details you're curious about, I'd be happy to elaborate.
What do you use to ship the logs into ES? I'm not familiar with the technologies you're using, but is this in-sourced developed tools? Or, are there OS options for log aggregating in ES that you would recommend?
Maybe not the best place to ask, but are you guys hiring summer interns?
There was one line that really surprised me: "We personally respond to almost every one of [the messages], without using response templates." Is that a feature? I think you can sound fairly personal even while using templates, and they could save a lot of time (30 seconds saved per message * 20k messages per month = 1 full time employee).
