How to generate reports?

What is a viable solution to generate SLA % reports for a SaaS service on cloud?

• As we offer a SaaS solution on AWS, of course we must include a SLA % on the contract with our customers, reporting about availability of the application.   Our goal is to have in output Monthly/Quarterly/Annual uptime reports, with the option to "pause" the counter during maintenance periods. A solution can be hosted monitoring software on one of our Instances, or a Third Party SaaS service that can monitor our application (we use EC2 and multiple VPC, with single tenant environments). Basically a SaaS solution is preferred because it can be a "third-party controller" that can be trusted for our customer, but this is not mandatory.

• Answer:

  It's an interesting question and one I've thought about a lot. We have long used Pingdom and presented that data, all-in, down to the minute going back 5+ years at http://trust.echosign.com/  I felt this was the right way to go. Until recently, there was no distinction for maintenance periods. What I've learned is this is definitely the right, customer-centric approach.  But no one does this.  No one else provides this 100%, multi-year, to the minute approach to service levels.  No one.  So maybe this isn't the way to go ;) Instead, everyone that does this at all in SaaS uses (x) vague self-generated system checks, (y) rarely provide true longitudinal data -- and most relevantly, (z) often hide smaller intrerruptions as being below the tracking criteria. For example, for many years, http://trust.salesforce.com did NOT count a service interruption of < 15 minutes as an event.  In fact, I'm still not sure they count it, now it just gets a vague soft exclamation point, which you can see below in this chart.  To me, this is pseudo-transparency, really marketing fluff disguised as a pseudo-uptime chart: You can argue both sides I guess, but by using Pingdom data or a similar real-time uptime source as your system-of-record ... you can't hide or fake this. On the other hand, I don't think a single customer has every acknowledged the value of this true transparency vs. pseudo-transparency of most of our colleagues.  So think about that ...