On the 12th of October I published a post about my experience with hyperconverged storage company Maxta.
The blog was an honest reflection on what turned out to be an expensive failed project for the cloud service provider I was employed by at the time.
I'm a big believer in learning from your mistakes, so sticking to the facts, I laid out clearly what had happened. The mistakes we had made as a team, (and there were plenty), were plain to see, as were the issues we had experienced with our hardware and the Maxta software.
I tweeted it to my 10 or so twitter followers, and shared it with the 150 or so connections on my LinkedIn. It was shared by two former colleagues of mine who still work for the organisation, @grump_tech on twitter and Ollybee on Reddit.
I thought nothing more of it. By the end of Friday, it had about 25 views, and I went off for the weekend to do normal non-IT type family things.
On Sunday night, Ollybee sent me a message to say another reddit user had reposted my blog to the Reddit sysadmin forum. I checked my website analytics and was genuinely shocked to see it had exploded from 25 unique visitors to 7,500 unique visitors. (I had also accrued a handful of additional Twitter followers and some LinkedIn requests)
Many of the commenters on Reddit felt that we got what we deserved with the way we the project had been run. I agree with them in many ways, although I have the insight of having seen first-hand the complex factors that were at play internally, and how they affected the decision making process.
Yesterday afternoon, one of my new twitter followers kindly tweeted me to tell me the register had picked up the story, complete with a statement from Maxta. You can read it here.
Maxta appear to have decided, that rather than take the medicine as we did, and learn from the experience as we did, they would rather dig themselves deeper into a hole by indulging in a game of post-factual press releases.
Strangely enough, I don't appreciate being falsely accused of lying, so I'd like to address the content of the Maxta press release here, section by section.
The customer contacted Maxta to evaluate our software. Maxta provided access to our then generally available release. The customer tested the software and determined it did not meet their requirements. They then disengaged to pursue other solutions.
Reality: This largely tallies up with my original post in terms of looking at their remote POC lab in March 2015.
Several months later, the customer contacted Maxta requesting to partner with us to deliver a solution that would meet their requirements. We offered to demonstrate an early version of a future release to determine potential fit. The customer stated the future version was exactly what they needed. Maxta stated the version was not ready, however, the customer insisted on testing it.
Reality: Maxta showed us a demo of the future version when we originally contacted them in March 2015. It looked promising and we told them we would be interested in looking at the product when it came out. In June 2015, the then vice president of sales at Maxta, came to England and took us out for a meal in Leeds with our UK reseller. We liked him and he made it clear that he was keen to maintain a relationship between the two firms. At the meal, he made it clear that the new version was imminent, and that he would like to arrange for us to have pre-release access as soon as it was ready.
Maxta furnished a beta version for testing with the explicit caveat that the software had known issues and could not be used in production.
Reality: This is the pre-release version of v3 Maxta that we suffered catastrophic cluster failure with. I made this clear in my original post. (Hence why we accepted it when it failed.)
Subsequently, without Maxta awareness the customer purchased a software license via a channel partner. When queried by Maxta about the reason for the unexpected purchase, the customer informed us they had deployed the beta version of the software to production.
Reality: 100% false. Maxta thanked us for testing the pre-release, and recommended we deploy Maxta v2.4. In an email dated October 16th 2015, they said the following:
"Our current production GA release is MxSP 2.4 which is stable and customers have deployed in production for the past 3 years. If you desired we could provide you that release of the product. 2.4 would be missing some attractive features that exist in the product you are currently testing but it is rock solid and stable."
They then sent us a link to download v2.4 and a 90 day trial license.
We installed it and tested it, as per my original post.
Which makes the next line of their statement particularly strange.
At no time did the customer deploy a generally available release of the software.
Reality: Either Maxta lied to us back then, and v2.4 wasn't a rock solid production release, or they are lying now.
After the problems we experienced with v2.4, we deployed v3, which they told us was now a GA product. I explicitly raised my concerns regarding v3 with them, and on the 27th January 2016 they sent me an email that started like this
"Thanks for letting us know your progress of preparing your cluster for our v3 version of Maxta.
In response to the concerns that you raised, we have addressed all the issues that you saw during your earlier testing. The V3 GA release went through extensive amounts of testing, including longevity testing, stress and limits testing, and hundreds of node and drive replacement testing."
On Feb 13th 2016, Maxta's technical team installed v3 on our cluster. (@grump_tech has put a screengrab of an email from Maxta confirming the install was successful here)
So again, either they were lying to me then, and v3 GA hadn't gone through extensive testing, or they are lying now.
Despite the surprise the customer had deployed beta software to production, Maxta made all possible efforts to address the issues reported which were primarily related to third-party storage controller firmware.
Reality: It seems odd that Maxta were surprised by either the v2.4 install given they had sent us the installer and license, or by the v3 install given it was performed by their own technical team. As per my original post, there had been issues with the storage controller running v2.4, and Maxta's technical team did diagnose this. However these issues were resolved prior to moving to v3.
While the author states they could not reach Maxta support, after checking our records we have not been able to substantiate this. Maxta has global 7x24 support that is reachable via telephone and email.
Reality: Perhaps they have forgotten this, but if they need reminding @grump_tech has two separate emails from Maxta in Dec 2015, in which they apologise for not answering our support requests during this outage.
The customer requested a refund from Maxta beyond the time frame and terms of the license agreement. As the customer did not purchase the license from Maxta the request for refund could not be approved. Maxta referred the customer to their channel partner. To date, Maxta has not been contacted by the reseller regarding this matter.
Reality: I couldn't tell you the specifics of the Maxta license agreement. All I know is we asked for a refund (as did our reseller), and Maxta told both us and the reseller that the product was non-refundable.
(@grump_tech has all the original emails still in case Maxta would like to further embarrass themselves.)
And there you have it. I wrote the original Maxta blog as a form of self-learning and catharsis as much as anything else.
I certainly didn't think anybody outside of those who already knew the story would read it.
And I certainly didn't think I'd have to write this blog.