“Decoding” the Gartner Magic Quadrant

Phillip: [00:01:19] Hello and welcome to Future Commerce, the podcast at the intersection of culture and commerce. I'm Phillip.

Aaron: [00:01:25] I'm Aaron.

Phillip: [00:01:25] Aaron? Aaron?

Aaron: [00:01:27] Aaron? Who's that?

Phillip: [00:01:28] Brian. My, how you've changed. Your voice is lower. You look a lot sexier. {laughter}

Aaron: [00:01:33] And yet... Ooh. Same fashion sense, though, I think.

Phillip: [00:01:35] {laughter} A little bit Costco chic we like to keep it on the Future Commerce podcast. Of course, Brian is not with us here today, but who else could possibly replace my venerable co-hosts? None other than Aaron Sheehan. The man, the myth, the commerce legend. Welcome to the show.

Aaron: [00:01:54] Thank you so much. I'm so excited to be here.

Phillip: [00:01:56] You're also my co-host on Decoded this last season. Catch us up a little bit on what's been going on in Decoded as we're coming to an end there of the second season. It's been pretty awesome.

Aaron: [00:02:05] It has been pretty awesome. We've had a great set of guests, I think, and really thought-provoking topics as we have unpacked how products come to life and how businesses make decisions about what they should sell.

Phillip: [00:02:19] And today on the podcast, we're going to talk a little bit about how businesses make decisions about what they should buy.

Aaron: [00:02:27] Whoa.

Phillip: [00:02:29] That is a segue. You can tell I've been doing this for a long time. Before we get there. Aaron, I just have a quick disclaimer, you do have a day job and you know, obviously your wealth of experience in the world of commerce, your reputation should precede you. But for those who aren't familiar, give us a little primer on who you are and sort of what brings you to this point. And then I'll give the disclaimer on the back side of that.

Aaron: [00:02:55] Sounds perfect. So today I am the Director of Competitive Strategy at BigCommerce, a proud Future Commerce sponsor, and sponsoring the Decoded podcast that we just talked so lovingly about. Prior to that, I spent many, many years in the eCommerce SI world competing against Phillip and Brian, actually working with Brian for a few years there we were coworkers for a bit.

Phillip: [00:03:22] It's true.

Aaron: [00:03:22] And been in technology for a much longer period than that and have had a lot of other weird random jobs. I did not get here on purpose as part of a deliberate career path. I just sort of like fell into it.

Phillip: [00:03:32] I don't know how many people actually chose this career path, now that I'm sort of thinking about it. But that's how we know each other, and that's how you sort of became a bit of an authority around what today's topic is. And just a quick disclaimer. This podcast is not brought to you by BigCommerce, but we are long-time friends and rivals in the ecosystem. I think it gives us really interesting, contrasting perspectives on how the world works around the world of eCommerce. Today, I want to do a deep dive on something that seems a little opaque to a lot of folks, especially depending on where you sit in the eCommerce ecosystem, and that is the analyst ecosystem and the reports that they produce and specifically kind of hinging this on a recent annual report that comes out from Gartner, which is a market research firm, and Gartner's annual Magic Quadrant for eCommerce Platforms and Digital Commerce Experience Platforms. So I thought we could come to a little bit of how those things come to be, how we're supposed to interpret them. And you can give us a little bit of insight into how businesses that make these platforms wind up on lists like this. So we're going to go deep today. Let's jump into it a little bit. Depending on what your role is in your org, you may or may not depend on a report like this to help you make decisions around the software you buy. So let's talk a little bit about like who's a report like the Gartner Magic Quadrant for?

Aaron: [00:05:06] That's a pretty good question. So my experience even long before I was in commerce, but just in software as a service in general is that the Gartner reports are for general CTOs, let's say, institutional buyers, but it's software they're usually... So it's often the CTOs purview to make a buying decision. To get a quick feel for like, who should I send my RFP to? Are you a solution that is stable enough that I want to bother sending you an RFP? That's sort of like the first, easiest. I mean, and this is where like the graphic, the quadrant graphic itself, gets shared out on the internet, like within an hour of the publications being...

Phillip: [00:06:00] But it's not supposed to be or theoretically, it's not supposed to be.

Aaron: [00:06:04] It's not supposed to be. But yet, there it goes.

Phillip: [00:06:07] Not hard to find.

Aaron: [00:06:08] Not necessarily. If you're a Gartner Quadrant or if you're a Gartner customer and you participate in the licensing and reprinting of the reports, you are allowed to use the Quadrant image on a landing page to promote the actual download link to Gartner itself. So the image is actually available on the public web for anyone to see. So in theory, while you're not supposed to use it, it's also out there on the public web for use. And so what happens... What happens, Phillip, to anything that's on the public web?

Phillip: [00:06:40] {laughter} It gets a DMC notice takedown and it disappears forever. And that's what has happened in the past to some things. I guess the question here is there's a difference between who's authorized to distribute it and who has a license to distribute it, versus how many people may distribute it without the license and whether they enforce it or not. I think often these are sort of hushed, closed-door sharing file trading that happens with reports like this that maybe the folks it's not meant for get a hold of it. And then they sort of question how those things sort of work.

Aaron: [00:07:15] Yeah, that's totally fair. But as I said, the sort of like, "I looked at the quadrant image and now I know who to send my RFP" is sort of like the...

Phillip: [00:07:26] The worst way to interpret it, maybe?

Aaron: [00:07:28] The worst way to interpret it, and the next best way to interpret that is the customers, the buyers, the CTOs of companies who are Gartner customers and who have calls with consultative engagements with the analysts who were part of putting the quadrant together. And that's when you'll usually get into the quadrant was really more lead gen for a consultative conversation where you're actually that analyst is going to get into the details of what's the right technology solution for you, buyer who I'm speaking to, who is paying me for this consultative engagement. And then all of the scoring and the evaluations and all the vendors sort of put into responding to the quadrant pay off for the buyer, because what they then get is a much more nuanced vision of what those platforms and solutions are actually good for.

Phillip: [00:08:20] Okay, so just taking stock already. Number one, this is sometimes looked at as a lead gen piece to a larger conversation for a type of a software buyer that a vendor is looking for who may be in the active process of shopping for a solution.

Aaron: [00:08:40] Yes. That is fair.

Phillip: [00:08:43] Is that a simplification? That's one person who might look at this report and say, "That's for me."

Aaron: [00:08:48] Right. Right. It would say that as a vendor, if I put my BigCommerce hat on, all of the platforms you see on that, we don't pay to be included on the quadrant. It's one of the things I do want to emphasize.

Phillip: [00:09:05] Say that again. Yeah.

Aaron: [00:09:06] We do not pay to be included on the quadrant.

Phillip: [00:09:09] Okay.

Aaron: [00:09:09] It's not paid. It not pay to play. The analysts have a methodology around evaluating solutions that determines whether or not, like who's going to show up, and there's criteria around like revenue and size and growth is a big part of it. They're not interested in anybody that's stagnant. They're not interested in shrinking platforms. They're looking for viable businesses. The quadrant evaluates the business, not necessarily the technology. And so they send out a big RFI early, early on, which is like, "Are you growing? How much? Where?" And then from that the analysts feed that data into the methodology machine. And out the other end comes a list of like, okay, these are the companies that we're going to include. So it's not pay to play in that...

Phillip: [00:09:57] Methodology. It's like the mystery van. It's like the Scooby Doo... You've got the gang is in there and the methodology machine.

Aaron: [00:10:06] Yep. And snacks. Snacks come out the other side.

Phillip: [00:10:09] You're ripping the helmet off and it's like, "Oh no, it's Mr. Jones all along."

Aaron: [00:10:16] Yes.

Phillip: [00:10:17] That platform that we all knew was...

Aaron: [00:10:20] It's an Excel spreadsheet all along is the methodology.

Phillip: [00:10:24] Yeah. Exactly right. So I'm making a joke, but I do want to come back. I want to push back a little bit, because in my former role as a VP of Commerce at a big global consulting firm, which that title is kind of meaningless, just for what it's worth.

Aaron: [00:10:44] It sounds really impressive, though.

Phillip: [00:10:45] It sounds so good. My wife thought it sounded really impressive. I was like, "Yeah, I'm an L10." But it doesn't mean... Out of how many Ls, Phillip? I don't know, like a thousand. So in that role, part of my job was to deal with analyst relations. So if you've ever dealt with public relations, if you know what PR is, AR is a very similar analog in the analyst world. So if public relations is to shop stories to the press to create pitches and try to garner attention from the press and to get publicity, then analyst relations have a very similar capacity where they are an agency that you can contact, or individuals that you can contact, that help put you in front of analysts so that you're on the radar, at least in the agency ecosystem. I have to believe that there is a similar capacity or function in the software vendor ecosystem to be in front of analysts, too. So while it's not specifically pay to play where you're going directly to the market research firm, like a Gartner or Forrester, and you are doling out dollars to them to participate, there is a middleman which serves as a marketplace who you do pay money to, who has the relationships, and they act as a bit of a gatekeeper. Is that inaccurate?

Aaron: [00:12:14] Well, it's accurate. I know of whom you speak. I will say that it is not a requirement to use those middlemen. And in fact, many of the analyst firms actively discourage using those and prefer a direct relation between the ISV... {phone rings} And I'm sorry.

Phillip: [00:12:37] That's the middleman calling now.

Aaron: [00:12:39] That's the middleman calling me now for their cut. So yeah, those firms do exist. They sell a very specific sort of "Hey do you want to represent well for analysts in general, but you don't have the time to actually talk to them? Call us and we'll we'll handle the PR, we'll handle the briefings," and stuff like that. To be fair, we don't do that. I do my analyst briefings myself at BigCommerce. And I've also done them in the past at other companies. So I've never engaged. So I can't speak to the value proposition or lack thereof of those other analyst relations firms, but they are by no means a requirement to show up.

Phillip: [00:13:22] Got it. Okay. Thank you for for setting me straight on that.

Aaron: [00:13:24] Absolutely.

Phillip: [00:13:25] So is there any myth busting here? So we've we've talked about like the pay to play aspect, specifically not pay to play, at least a couple of the big ones that we've probably mentioned by name already here, talk about the way that the quadrant, at least the Magic Quadrant from Gartner gets distributed and interpreted and probably unfairly, you said it's lacking nuance. What are some of the nuance that gets lost in that 2x2? And if you're not familiar with it, we'll link it in the show notes to at least one. We'll probably mention a bunch of things on this particular episode. But there's one graph in particular that was put out by Slava Kravchuk, a friend of ours from Atwix, which is an agency that has traditionally competed more in the open source space. I'm not sure what specifically they do today or what platforms they align themselves to today, but they did this like three year sort of motion or dynamic graphic that shows you where platforms started and where they ended over a three year period from 2021 to 2023 on that 2x2 and on the 2x2 and Gartner, the vectors are completeness of vision from left to right and ability to execute from top to bottom or from bottom to top, where the top right most quadrant is leader and the bottom left most quadrant is niche. So let's talk about that a little bit, which is the 2x2 is insufficient for interpreting how you might go about selecting a solution. What's in the report that then gets into deeper specifics?

Aaron: [00:15:14] Yeah. So I think you just stated the axis, the vectors, the ability to execute and completeness of vision. So for commerce specifically because Gartner puts out lots of quadrants for lots of software solutions. There are also lots of other analyst firms. Forrester puts out a wave, which is a different methodology. IDC puts out a marketscape. Everest puts out a peak matrix. There's always there's some sort of cute branded name that lots of these analyst firms have that is effectively an attempt to flatten what is an enormous amount of complexity into a two dimensional graphic, which if you're looking for a place where AR or like 3D images or a JavaScript library could really improve analyst relations, I think it's taking the flatness of this and adding more dimensionality to these reports would be really interesting. The first analyst firm to do that, I think, is going to clean up because of what you just alluded to, Phillip, which is that it's taking something that's very, very complicated and simplifying it down to basically running it through a methodology, sort of a standardized methodology. Because you talked about Slava's graphic, which is sort of the three year progression of platforms and as they move around, that only makes sense if you have a coherent methodology that stays set from year to year.

Phillip: [00:16:40] {laughter} Right.

Aaron: [00:16:41] So if you update your methodologies to allow for changing market forces or changing buyer behavior, you run the risk then of being unable to make comparisons. And it gets fuzzy maybe a little bit like, "Well, the criteria changed. Somebody's a winner, somebody's a loser." Everybody's mad, except for the new winner. And so I think these methodologies get a little calcified probably over time, but that's by at some level design. They're not meant to be sort of continually updated because they're meant to be a point of comparison year over year.

Phillip: [00:17:19] There are a couple of things here. One, if Gartner or Forrester or whomever had a completeness of vision rubric, let's say they understand every single vendor's vision, their roadmap, and where they are on their progression of roadmap, then in reality, every point on this quadrant, at least on the Gartner Magic Quadrant, is not relative to each other, but relative to their own product roadmap. Is that how I'm understanding you?

Aaron: [00:19:04] Yes, they're all evaluated. So within ability to execute, there are, I think like 6 to 8 components that make up ability to execute. And they evaluate things like the product, but they also evaluate things like financial viability, pricing, market perception, customer experience. That all makes up sort of that ability to execute for them. So again, I think a lot of people will take this and say, "Well, we're number one, which means we have the best product." That's not actually the way to read the quadrant. And I do want to call something out. We're talking about the Magic Quadrant. Gartner has a point of view and a methodology to how they produce their report. It is not equivalent to what Forrester does with the wave or what IDC does with the Marketscape. Each analyst firm has a point of view on their methodologies as to how their reports and the attendant graphics should be consumed and understood. So I do want to sort of caveat that and say the things that we're saying, I'm using Gartner as an example. They're not necessarily also reflective of the way a Forrester analyst would think of how they would want to communicate the value of the wave, etc.

Phillip: [00:20:24] There's a challenge here that I'm going to be really careful around. So I'll say some things that maybe you can't really build on understanding your day job role, which we'll do our best to try to not make you have to represent here on this show, because I'm just trying to understand this. When you see dramatic movement, like for instance Commercetools. So Commercetools this year stayed in the leaders quadrant but moved up and to the left. So one really, I don't know, one oversimplification here could say if all things were held equal from last year to this year, they became less complete in their vision. But became more able to execute? That doesn't even quite make sense unless the scale, the total scale of the completeness of vision changed dramatically over time, in which case you could say if their vision increased in total and their roadmap for their product increased in total, then therefore their less further along in their roadmap than they were last year. So that warrants the movement to the left in relation to where they were last year, which is hard to get an apples to apples to everybody else's. And in some way, you could basically make the argument that none of these should be viewed next to each other. None of the dots should ever be looked at next to each other. They should be held in isolation to understand which companies have higher product velocity and longer term vision around where their product is going.

Aaron: [00:22:13] So a couple of things I think and I'll be as... I am under obviously have an NDA sort of because I've worked for a company who's a Gartner client. So what I'm saying is I can't share with you the text history and tradition of the Gartner Magic Quadrant. What I would say is, yes, up to a point, the unexplanation for a company going backwards on completeness of vision would be a perception that the company has contracted what it views as its total addressable market. I will give you a hypothetical example. Kibo. But I'm not going to make any comments about Kibo's place on the Magic Quadrant. God knows we talked about it enough when the Forrester wave came out last year. But the private equity company that owned Kibo also owned Monetate. And they were sort of bundled and sold together as a comprehensive personalization and digital commerce solution. Since then, Vista Equity sold off Monetate, and it is no longer bundled with Kibo. So from a total addressable market completeness of vision piece in 2022, the Kibo Analyst Relations Team might have said, "Our vision for digital commerce includes full a/b testing, personalization, experimentation, marketing automation as a coherent part of a coherent platform with the digital commerce, the catalog management, the PIM, the order management, the checkout, and all the rest of it." Then a commercial event happens and almost definitionally you would expect to see Kibo move left as a result of their own choices.

Phillip: [00:24:11] Right. Got it.

Aaron: [00:24:11] Now that is an example. Other reasons you might see for that is what you were alluding to, which is the analyst used what was the bar for being innovative in 2021 is not the same bar to be innovative in 2023, which I think we want. Right?

Phillip: [00:24:32] That's right.

Aaron: [00:24:32] We don't necessarily want "Oh, you built a Rest API" still being like "yay" in 2023 the way it might have been like kind of new in...

Phillip: [00:24:43] 15 years ago. {laughter}

Aaron: [00:24:45] Yes. I was trying to... Every time I try to do math on that, by the way, I'm like, "Oh, 15 years ago, like that was the 90s. Right?" And then I have to remind myself that I'm old. Anyway.

Phillip: [00:24:56] {singing} "Old guys in commerce"

Aaron: [00:24:56] {singing} "The podcast"

Phillip: [00:25:01] Okay, so all of that makes sense. And I think hopefully everybody's tracking because these are conversations that we just don't have enough of, and I'm like, "Gosh, somebody needs to make like the definitive piece of content on how all this stuff works." So that's really interesting. In which case, I think then a Magic Quadrant doesn't have the same sort of illustrative quality to it, to me that a wave does, which means that technology cycles come in waves. The understanding... Your face right now. The understanding that as there are ebbs and flows in the way that we enlarge our vision and ways that technologies get applied across large portions of an ecosystem. And so certain things are possible today that weren't possible ten years ago. We all understand that. And as those things become possible and businesses start to move towards putting those into their products. You said marketing automation, right? Marketing automation was like revolutionary 15 years ago. So now that sort of table stakes. So it does make sense to me that the scale of the graph changes over time. And therefore people that haven't had lots of forward progress tend to be perceived as moving backwards. It's really that the scale of the graph is changing over time. And that's a really interesting intuition because it's not about movement within a fixed scale, it's that the scale is constantly moving in both directions and your velocity as a business and your total addressable market as a business determine where you stay on that stretching canvas. Right?

Aaron: [00:26:56] Like a lot of human endeavor, the analyst reports are an attempt to impose a scientific rigor on what is often a somewhat emotional set of judgment calls. And so we build methodologies, we build spreadsheets and matrices, and we relentlessly quantify things almost in... A larger philosophical point, I think this is a tendency in modernity to sort of try to atomize and quantify as much human experience as we possibly can into a mathematical scale, so as to remove human judgment. And so, like, "It can't be wrong. I was just following the math. Just believe the science." But there's always that core, to your point, of subjective human intent and goals. And sometimes it may be subconscious. There's got to be a value add. And the thing that's ironic about this to me is consulting. When you pay for a consultant, Phillip, you're paying for their judgment. If all my business problems could be solved by appealing to a calculator and a macro, I would not need to hire a consultant. I would just pull out my calculator and Excel and solve them. And so there is always that tension, I think, between the sort of I don't want to call it faux rigor because there's a lot of rigor that goes, I don't know, I haven't seen the model, but I believe there's rigor that goes into these evaluations and these models. But you're absolutely right in that the velocity of one platform is somewhat not under their control in the sense that the bar for where you move up, the unconscious gradation of like what is innovative and what is not does change every year.

Phillip: [00:28:38] I have to believe though, and this is an interesting... Let's talk about incentives for a second, because I think that you could say that the existence of these reports or market scapes or whatever, this sort of market research levels an incentive on the platform to try to perform well or to be perceived as performing well. And so why wouldn't you just build the things, I mean, this sounds so simple... Why wouldn't you just build the things that are required to perform well on the platform, at least in on one of these axes, it looks like you could just have a really great product that has lots of features, functionality and completes a lot of the things that everybody needs. Why would you ever be to the left then? What are the forces that keep you from performing really well, holding all things equal, like if it's able to be quantified, to your point, what we need to do in order to win, why wouldn't you just win? Seems crazy. It seems easy.

Aaron: [00:29:52] You should coach a football team, Phillip. "Hey, who wants to win? Just go do it, guys. Super easy."

Phillip: [00:29:58] Like freaking Deion Sanders right now. Like, I could do that. I could do that locker room speech.

Aaron: [00:30:03] Prime Jackson they call it.

Phillip: [00:30:05] Prime Jackson.

Aaron: [00:30:06] That's right. I could see with some real gold jewelry. I think you could pull it off.

Phillip: [00:30:11] I could pull it off, I think.

Aaron: [00:30:13] There's a complicated answer.

Phillip: [00:30:14] The shades.

Aaron: [00:30:14] There's a simple answer to your question and a complicated answer to your question. Which one do you want?

Phillip: [00:30:19] Give me a simple first. It's the end of the work day. {laughter}

Aaron: [00:30:22] The simple answer is that winning the Magic Quadrant does not equal commercial success, and most companies are optimized for commercial success, and they are not the same thing. And there are plenty of platforms that fall off the quadrant. And so just because you showed up doesn't mean that makes it rain. And so yes, a company can absolutely. I think, try to game the quadrant. My personal opinion is that is exactly what Magento did for a while. They hired someone from an analyst firm to run their product vision and they started climbing the analyst reports. And I remember a period of a lot of innovation. This was back when it was, you know, Premiera I think was sort of the when it was privately held and that that interregnum between eBay and Adobe when Magento released feature after feature after feature, including cloud hosting. And to be honest, most of those features were super half baked, and a lot of them never got wide adoption. But boy, it made them look good on the quadrant. And so I think what you're saying is exactly possible. However, that's not a company's only goal and highest goal. And so doing that kind of thing, chasing clout costs. And because what your customers are telling you, they want you to build is not the same thing that analysts want you to build. That's the simple answer. The complicated answer is nobody ever talks about it. But the Magic Quadrant comes out with another report called The Critical Capabilities.

Phillip: [00:32:07] Oh.

Aaron: [00:32:08] Yeah, I know. No one notices it.

Phillip: [00:32:09] If it's so important and it's called Critical, why aren't we talking about it?

Aaron: [00:32:13] That's a great question. I don't have an answer for you, but I will tell you that The Critical Capabilities, it's often...

Phillip: [00:32:19] Because there's not a chart. That's the answer, Aaron. There's probably not a chart. There's no chart.

Aaron: [00:32:24] There's no sexy presentation. That's correct. That's correct. What there are is there are a lot of Excel fields. And you can go out and you can get The Critical Capabilities, and that's where they say, "Okay, we're going to evaluate. We're going to show you the actual product scores for these platforms we went through. We, the analysts, went through and scored everybody across a variety of use cases. We looked for examples. We called references and we got scores." That's a separate report right. But that report is actually the report that people think they're looking at when they're looking at the Magic Quadrant. I made this mistake for a really long time. Because to your point, you only ever see the graphic. We like pictures, we don't like spreadsheets.

Phillip: [00:33:05] That's right.

Aaron: [00:33:06] So we read the picture, we look at the picture. And so what I will say is if you looked at the quadrant right. And you see that visionary side of the quadrant. That's that bottom, right. That's where the completeness of vision is the strongest. But they're smaller. They haven't found the commercial success. So you could say that the companies that have done a good job in doing what you just suggested, which is like not gaming, but like building out stuff that analysts want to see either on purpose and in the purpose of fulfilling their vision, that's where they show up. And then ideally for them, they rise up the ranks. Oro's over there with everybody's good friend, Yoav

Phillip: [00:33:55] Yoav Kutner. Yeah.

Aaron: [00:33:57] We just speak in shorthand now.

Phillip: [00:33:59] Yeah. I'm just giving context for folks. Yeah. Oro Inc, which has been around almost a decade now, I think, which was like a B2B centric platform. It was like former Magento Co-Founder who was like, "I'm going to have a second crack at this." Really interesting open source roots too. Again, not to like nitpick what's on this 2x2 because I think we already said that it's, you know, maybe problematic just to look at that. I find it really interesting that I have heard rhetoric for years that platforms like Shopify don't necessarily play this game because they determine maybe it's like not really worth their time or effort or they've just not been in the analyst space, mostly because they've been focused on a different market, which doesn't require analysts quantification in order to be successful or commercially successful. So if that were true, which I guess you can correct me here, but if that were true, how can an analyst chart something without a platform's participation? How much do you have to participate as a platform in order for the analyst to make some calculation about these factors?

Aaron: [00:35:21] Really good question. I will say that I don't know that it's still true today that Shopify does not participate in these because of their go-to-market changes recently. I would be surprised if they weren't playing the game because they are now aspiring to that same... If you're going to market now with Deloitte and KPMG and large global companies, probably this is a good idea for you. But I don't know. What I would say, though, is that often if you look at the finished version of any of these analyst reports, companies that don't participate basically means they don't respond to the RFI. The RFI goes out months in advance of the actual report coming out, and you have the opportunity, "Hey, you've been selected." Just because you got the golden ticket to Willy Wonka's factory doesn't mean you have to go.

Phillip: [00:36:13] It doesn't mean you get to turn into like a purple...

Aaron: [00:36:16] You don't have to turn into like a jawbreaker or a gobstopper. Yeah, exactly. Fun fact: my first play that was ever in as a child was in Willy Wonka's Chocolate Factory, which was the worst casting, I think I can imagine.

Phillip: [00:36:29] What was your role?

Aaron: [00:36:30] Augustus Gloop.

Phillip: [00:36:31] Well hey...

Aaron: [00:36:32] Very large. And I'm not a large, and I was not certainly at the age of ten.

Phillip: [00:36:38] I mean, hey, kudos to them for not typecasting.

Aaron: [00:36:42] Yeah.

Phillip: [00:36:42] On a completely unrelated note, one of my favorite bands in high school was Veruca Salt. So there you go.

Aaron: [00:36:47] There you go. Oh, nice. Okay, speaking of, {music} "Old men of commerce..."

Phillip: [00:36:51] {laughter} Yeah.

Aaron: [00:36:53] {laughter}

Phillip: [00:36:55] Wow.

Aaron: [00:36:56] Then I was never in another production after that, and that's why. And now I'm healing. I'm healing from that trauma through this show.

Phillip: [00:37:03] This is a safe space.

Aaron: [00:37:04] This is a safe space.

Phillip: [00:37:04] All your traumas are safe here. Nobody's listening to this. It's totally fine.

Aaron: [00:37:08] I feel that, I feel that, thank you. I feel that. Whatever the point was, boy, I've lost it.

Phillip: [00:37:14] Well the point was, you may have gotten the invitation, but you don't necessarily have to accept it.

Aaron: [00:37:18] Yeah. And so what you'll see often on the analyst reports is when there's a vendor who is sort of invited but doesn't show up, a little asterisk by their position, it'll say like, "We, the analysts, are the best we could ranking this company based on publicly accessible information." You didn't give us references to call. You didn't present your roadmap. So we don't know what you're going to be doing. But we can look at your case studies, your financial health. If you're publicly traded, we can talk to other people in the ecosystem who have a relationship with you and gather sentiment. We can look at dev docs. We can, you know, all that stuff is available to them, and then they'll score you based on that, but it always comes with sort of an asterisk, which is like a "Yeah, we don't have as much access to this." I will say, and I will be blunt, I think it probably hurts you not to show up and participate in the RFI because you lose the chance to present your vision and your roadmap and cast yourself in the best light to the analysts. So that's a good thing for you to do if you are wanting to rank well.

Phillip: [00:38:57] Is it possible that working with analysts, and and caring about this sort of thing is a bit of a distraction or creates a different type of incentive for a platform, especially early stage, where there are some specific types of things that you may need to overcome that you might otherwise become enamored with. You gave a couple examples earlier, but there are things to probably have to do that requires a lot of time and probably pressure in the organization to have to produce some case studies or put some clients on the line, ask them to participate on for you. Maybe that creates other problems where there's a little bit of horse trading in that you probably owe them something and they now you're asking them for a favor or I don't know. Those all seem like challenging things to have to overcome that all require people dynamics, not just raw data. It requires people to participate. And that takes time. And those things, you know, mean you have to... It's a distraction. I don't know.

Aaron: [00:40:54] Well, I wouldn't characterize it that way because I am paid a salary to do this.

Phillip: [00:40:58] That's your job. Right.

Aaron: [00:40:58] Like if I come out and agree with you, then I have to be looking for work.

Phillip: [00:41:03] Unfortunate. {laughter}.

Aaron: [00:41:03] If you're successful here at selling this vision. But no, I think of it as sales. It's sales to a different party and the value proposition. It's any kind of... You've been through RFIs.

Phillip: [00:41:03] Oh yeah.

Aaron: [00:41:22] One of the things that's clarifying in an RFI process and I've got one I'm supporting actually at this moment. But it is a forcing function to make you organize your information and clarify your thoughts around what is your vision, what is your strategy. And a lot of times people know it intuitively. They have a version of it in their head. And of course when you're talking to a software company, there are like 1400 of us at BigCommerce, roughly around that number. Other platforms have more people, other platforms have fewer people. But wherever you have more than one person, you have different opinions and thoughts about what that stuff is. And so it does force you to sort of like get that stuff together and organize it. And often there are good things that come out of going through the process beyond just the ranking on the quadrant. It's always good, I think, to force collaborative cross-functional team engagement. That's what this is, right? Product, sales, finance, product marketing all have to come together to sort of like put... Because the RFIs from these analyst reports are quite large and quite thorough. And you all have to get together, you have to share information. And it's you could call it trauma bonding I guess. But I think it's good for you overall just as people as a company, as a business to go through the process. It is a lot of work. I don't want to downplay that. It is a lot of work, especially when you have, you know, 3 or 4 a year of these hitting and everyone requires its own project management and program management and oversight and all the rest of it. But yeah I think it's valuable. And I think, on the commercial, the grubby commercial side of it, Phillip, anybody that shows up relatively well on the quadrant gets RFPs at the end of it because as I said at the beginning, this is validation.

Phillip: [00:43:24] Yeah. Theoretically. It should pay off. Right.

Aaron: [00:43:27] Exactly. If you're on here then a fit for commerce or a similar sort of shop will include you on certain evaluations. A CTO will think, "I'm not taking a big risk if I send my RFP to to these companies." Not because the quadrant is an answer, that the picture actually answers the question, "Is this the right solution for me? It was the top and right most platform, in a general evaluation, therefore, it's the right one for me." Because if you look at the quadrant specifically, I'm looking at the graphic now, and I see giant suites like SAP, Salesforce. I see headless only platforms like Commercetools and Spryker and Elastic Path. I see open source platforms like Ora that we just talked about. I see ERP bolt ons like Sana. I see multi-tenant SaaS platforms like BigCommerce and Shopify. This is a graph saying vehicles. "Here are a bunch of vehicles." And if I'm shopping for a car for my family and I look at this, there are trucks, there are electronic vehicles, there are plug in hybrids, there are SUVs, maybe the occasional tank on here. And it's like, well, if an Abrams tank is the top right most vehicle on the vehicles quadrant, because it could do the most things and like shoot a big gun like it's the best vehicle for a family. Not necessarily because the sprinter van is also on here or whatever you want to sort of like have. And so that's where the nuance comes in.

Phillip: [00:45:15] A different way for me to think about how to look at something like The Magic Quadrant is what we're mostly looking at is flat plane, two dimensional, static graphics. But to really understand, you kind of need something that's interactive, that's based around a lot of criteria that might apply to your context. So if you're to interpret this, one way to do that is you read the full report on it in its entirety, right? So you can come away from looking at and interpreting that report and you'll have your own context. You can apply to whether something that you're already shopping for may be fit for you or may not, or you can go and yeah, I guess the deficiency and maybe some of the critique that's often applied to these analysts reports is the nature in which the visualizations are delivered. And maybe an evolution of that would be to get an intuition of how the report actually works, you need to be able to play with some of it. And that would require you to have some insight into the inputs that make the report what it is. Which is like a proprietary thing on its own. So I guess I'm just more musing around there are ways to dispel some of the criticisms around the report, but the people I guess generally that are criticizing the report are not the audience for whom the report is meant.

Aaron: [00:46:58] Maybe, maybe. I mean, there's certainly a fair amount of criticism from buyers about the report. We may link to some of that in the show notes. But I do think it's fair to say that if you are a buyer of software and you don't already have an institutional account to Gartner or Forrester or IDC, or one of the firms that produces these so that step two of reading the report is talk to the people who wrote it. The value proposition of simply looking at the graphic and trying to make a decision is actually pretty low, and you're probably better suited to actually talk to a consultant in the space who can walk you through a transparent methodology of selecting a platform for you because just buying the report and then never digging into the raw data behind it, I think, is actually not helpful for most people, most companies. And so if you want to know, I would say maybe call someone, maybe call Future Commerce, maybe Future Commerce could help you decide what platform you should be on.

Phillip: [00:48:10] That sounds like an amazing offering.

Aaron: [00:48:13] It does.

Phillip: [00:48:14] Yeah. Call Future Commerce. You can do that actually. Sales@FutureCommerce.com. Okay.

Aaron: [00:48:21] Brian will be so happy. He's with us in spirit.

Phillip: [00:48:23] Oh my gosh, Brian. Brian is frothing at the mouth wherever he is right now, it's like {sound effect} Yeah. He and I have very different perspectives on how you could possibly even give someone advice about what platform to choose these days. I think I would relate that to also having had a conversation with somebody around content creation, they're like, "What's the best podcast platform?" And my first question is, "Why are you thinking about podcasting in 2023?" {laughter} If you're not already podcasting in 2023, my first question is, "Why are you thinking about starting a podcast?" Because there's no distribution in that platform. You have all kinds of other problems to then have to solve for beyond, just "I want to make a thing." Making the thing is one teeny tiny little part of the equation. And I think that has in my mind, at least my gray matter, which is very smooth these days. It used to be a lot more. I used to have a lot more gray matter. But, you know, the longer that I become a content creator, I think the smoother my brain gets. But that's neither here nor there. I just think to myself it's one thing to have a well-built eCommerce platform. It's another thing to be successful with it, which is like getting people to that platform to buy is actually also a gargantuan challenge. There's no context for that in the world in the Gartner Magic Quadrant. Okay, I did a little musing there. I do want to ask another question. I've heard rumors in smoky rooms that there is then a process that you've gone through the RFI, you've now had the analyst briefing. You've become quantified in some way. When do the platforms get wind of where they placed, and do they have any say so in sort of a rebuttal? Is there a period where the platforms can then give some more context, where context might be lacking as to why they don't like where they placed on the Magic Quadrant?

Aaron: [00:50:35] Yes there is. It's a very limited and short window because this is all on a larger commercial timetable. So typically, you'll get a draft of the report with only your information on it. So a graphic that does not have competitors on it. And it's like "Here's where you scored and what we're writing about you. Do you have any factual rebuttals to this?" Meaning, "Did we actually get something wrong?" That is what the period is there for. Editorial changes are theoretically possible. Obviously, quadrant placement is it is where it is. Because the analysts all have methodologies. As I said, they've got a machine. They've got a black box into which numbers have been put. And so like you have been stamped and numbered and you may say, "I am not a number. I am a free man." It doesn't work, doesn't cut much ice. But if they say something in their report that you can sort of point to and say, "Look, I sent you this. This is in our 10-K," or whatever. "Oh, sorry. We got the number wrong." Yes. There is an option to do that. It's a 1 or 2 week period usually that you have to sort of do that.

Phillip: [00:52:07] So that's interesting to me. And that's sort of like okay, you're in dialog with the market research firm. The Analyst. These analysts are in the role for a really long time. They generally stick around. Is that true?

Aaron: [00:52:26] Often. I feel like the tenure at Gartner has been pretty good. Forrester's been pretty good for commerce. IDC recently turned over Jordan Jewell, who is now working for VTEX. I think he left last year or maybe late 2021. And Heather Hershey, who is his replacement, started last summer. But yeah, generally speaking it is a longer term role and it makes a certain amount of sense. Because a lot of the value of the role comes from network and experience. And so it's definitely a role that you, I think, would go into after a fair amount of real world experience, not something that you would sort of pick up on as like a foot stool towards a different role usually.

Phillip: [00:53:16] So this opened up a couple of questions I really hadn't even thought of. How does one get a job as an analyst? What are the requirements to get a job as an analyst at one of these market research firms? What is their typical career path leading up to that role?

Aaron: [00:53:32] Great question. I've seen a lot of variation in the analysts that I've worked with. Some folks come out of our world, Phillip, some of them come out of the SI world. And they decide instead of going to product, they go into an analyst role. Because if you think about it, what we've all done in working at agencies and solutions integrators is evaluate tech and learn how to quantify the unquantifiable in our own proposals and delivery recommendations. So there's a certain amount of common skill set. A lot of people come out of academia.

Phillip: [00:54:09] That makes a lot of sense to me. Yeah.

Aaron: [00:54:11] Right. Yeah. Because because of that rigor piece that I was talking about. Sometimes you'll see people come out of larger sort of management consulting. So McKinsey or BCG background where again...

Phillip: [00:54:26] That makes a lot of sense too.

Aaron: [00:54:27] Yeah, exactly. But I would say it does tend to attract people with specific industry expertise in a particular set of tech. Because like I said, all of these firms evaluate lots of different solutions. And so there's an insuretech quadrant and a digital commerce quadrant and an ERP quadrant and all the rest of them. And so, you know, if you come out of that world and you have sort of hands on experience, and then you also have either the proper academic or consultative background, I think you can be a good fit for that role. I can put in a good word for you if you're looking.

Phillip: [00:55:03] {laughter} Yeah. I don't know, maybe there's a Future Commerce visualization at some point in the future that we could craft up. There's an interesting, so we actually touched on this earlier. But since we're on the topic of like how you get into the role of an analyst, I kind of laughed a little bit when you said it earlier, but I didn't like just point it out. It's kind of funny to me that analysts then often or with some with some frequency wind up working for the platforms that they...

Aaron: [00:55:41] One example of that like, yes.

Phillip: [00:55:43] Or a couple.

Aaron: [00:55:44] A couple.

Phillip: [00:55:45] Right. Yeah. Yeah. Okay. Yeah, that seems interesting to me.

Aaron: [00:55:50] The fox guarding the henhouse. Is that the phrase?

Phillip: [00:55:53] I don't know. Maybe. I mean, it depends on how often that happens before you start to say like, something's fishy here. I can't see that it happens very, very often. Like any merchant or retailer, we talked about this in the Decoded podcast, for a market research firm to continue to grow, they either have to court more customers, which means the market has to continue to grow itself, or they need to find more things to sell. And so to me, it makes a lot of sense that there's not just one digital commerce platform market scape. There's probably a growing number of those. What are some of those? I imagine there's some that are like this is for marketplaces, this is for B2B or some flavor of like industrials. Like how finely grained are they getting these days?

Aaron: [00:56:39] This is a pretty interesting question. So my earlier comment, maybe it was a kvetch about the Magic Quadrant being vehicles. And it's got all sorts of vehicles on the quadrant. You'll see some firms for instance are slicing it. They'll do a Forester, for instance, has a B2B wave and a B2C wave. That gives you a sort of a somewhat handy way of saying, "Okay, separating the sheep from the goats, basically." But then, of course, there will be vendors who play in both. And so like that's also useful information. I've also seen there are analysts, there's an analyst firm out there called Paradigm that puts out a report called The Combine every year. Andy Hoar runs that, ex- Forrester Analyst I believe. He did Death of a B2B Salesman, I think back in the day. And then now he's got his own firm. He does only B2B. His entire practice is consultative practice and his analyst publication is purely B2B. And he does a mid-market B2B and an enterprise B2B Combine. You will also see some of them that will come out. I think IDC does a headless one, for instance. Like so if you're a headless platform. And that's, I think, helpful in the sense that you can get very specific. And the more specific context you can bring in to your graphic, I think the better, the more useful your two dimensional graphic probably is, because that context is the actual third dimension in that sort of like 3D visualization that I was advocating for at the beginning. It's the context that helps you interpret the graphic. And I think that's actually a good thing for a buyer because it lets you understand at a glance kind of who the products are for that you see on the list.

Phillip: [00:58:29] Gosh, I wish... This is a complete non sequitur, but there is an incredible YouTube channel called, 3Blue1Brown. I'm not sure if you're familiar with it. It's a mathematics YouTube channel, but what makes it so powerful is unlike Salman Khan from Khan Academy, who does a really great job of showing you building up things like your understanding of things like vector calculus through writing, like as if it was a whiteboard. This person is a DataViz wizard and is really great with animation specifically like Python based, really complex animation to show you how these things work so you can see movement and motion and get an idea of how size and scale and things like, I don't know, again, like vector calculus. What I often feel we lack in this little ecosystem of eCommerce is that kind of expertise for folks that not just new to the industry, but even old timers like me. Unless you have to work with this kind of a piece of data or some sort of visualization of some kind of report over and over again, even just seeing the arrows showing it moving from one place to another doesn't really tell the whole story. It's a really interesting thing. I guess I wish that we had more of that sort of deep visualization of the kinds of things that are sort of trend centric, in this ecosystem. All right. One last, I guess, one last thing to sort of, kind of get us wrapped up and then maybe you can tell me things we may have missed along the way. How do you fall off the quadrant?

Aaron: [01:00:21] Well, good question. I think going out of business is a really good way to do it. So with most things, I think that's definitely true. I think I said at the beginning, there are criteria for The Magic Quadrant around things like size of business, revenue growth and the like. So, for instance, I noticed that on this Magic Quadrant, Oracle isn't here anymore. They are gone. Oracle is not out of business. I don't think anyone would say Oracle is out of business. And if you do you'll hear from one of their lawyers. But however their digital commerce offering I think it's fair to say it's dead. I mean, I don't know, Phillip. What do you think? What are people saying in those smoke filled rooms where you get your information?

Phillip: [01:01:14] There's a number of things that I think have been declared dead that are still very much alive on this quadrant anyway, which I find to be another interesting...

Aaron: [01:01:25] Separate, separate conversation.

Phillip: [01:01:26] Totally separate conversation. {laughter}

Aaron: [01:01:28] So, yeah, I think sort of like laying off all your people and no longer being commercially viable, effectively not showing revenue growth is what it boils down to. If you've contracted in terms of the piece of your business that's digital commerce. And now, look, I mean, for these larger suite based companies, that's always a little bit of a crapshoot, like SAP and Salesforce because none of them report earnings that are specific to just the commerce line of their business. And so there's always a little bit of sort of self reporting going on. But I think if if you fall off, that's one way to do it. I wonder another one has fallen off completely is Optimizely, which is really formerly known as Episerver, at least in terms of the former quadrant presentation. There was a weird sort of git merge thing that happened with them where Episerver acquired Optimizely and then took Optimizely's brand.

Phillip: [01:02:22] Objectively, they had the better brand. I mean.

Aaron: [01:02:24] Shout out, shout out to Optimizely OctoCon, happening right at this very moment. But that's one where they have, I think, as part of their acquisition, what it feels like from the smoke filled rooms that I'm in is that they have de-emphasized digital commerce in the last two years as a product suite, which is odd again, for Episerver, but as a result, they have not grown enough to merit being on the quadrant. So you don't have to go out of business. Sometimes you can just sort of take your eye off the ball to fall off the quadrant.

Phillip: [01:02:59] Yeah, or even decide that, yeah, take your eye off the ball. I think that implies some sort of like a dereliction of duty that's maybe not necessarily true but is more about you can't do everything in this ecosystem. I think the massive suite days are maybe numbered. It's the ecosystem really has room for, I got this from a recent conversation I had with some folks about how much room is there in any technology ecosystem for five, ten, 15 competitors? After some period of time, things kind of settle out to being 2 or 3 major competitors, and they take up the large amount of volume like AWS or cloud services being the greatest example is. There's AWS, Azure, GCP, that's it. Yes, there are smaller ones for sure. And there always will be because there's always room for Jell-O, I guess. But for the most part, what you have is a the pie is made up largely of a few big players, and what we're spending the last couple of decades doing is just determining who those big players are.

Aaron: [01:04:15] I don't know, I'll push back on that a little bit, because I think that that is true in some aspects, but that is the exception rather than the rule. I think if you looked at the fortune 500 or any large number of most valuable companies over the last ten, 20, 30 years, you'll see an awful lot of people who were there ten, 20 years ago not there today. And I do think that there are some realms that lend themselves towards, I don't know, is monopoly the right word, but a duopoly, a stable duopoly. Coke and Pepsi.

Phillip: [01:04:48] Oligopoly.

Aaron: [01:04:49] Yeah. But I think that with software we've mostly avoided that. You can point to in CRM, you can say, well, there's Salesforce. And that was true. And then HubSpot came up out of nowhere and just started taking market share away. And I think what happens is a lot of these companies build an audience where there wasn't one before. And this is a little bit, this is Clay Christensen's sort of like, you know, Innovator's Dilemma piece, which is a lot of companies will enter a world that is stably defined by a small set of enterprise competitors, and they will start taking those guys lunch slowly over time, and then they will become the thing that they were sent to destroy. Anakin. But that's a cycle. That's a creative destruction.

Phillip: [01:05:40] They were the chosen one.

Aaron: [01:05:41] They were the chosen one. Salesforce had the high ground.

Phillip: [01:05:44] I was going to say, you had the high ground. It's over.

Aaron: [01:05:47] We can do this all day.

Phillip: [01:05:49] Wow. From Willy Wonka to Star Wars, we really run the gamut. Oh my gosh. That's a really interesting insight. Thank you for the pushback. I guess it really comes down to how you see eCommerce or basically like shopping cart or catalog management software. Do you see it as differentiating software that runs some facet of the customer experience that demands its own brand and brand affinity from a software buyer, or do you see it as a commodity and plumbing that doesn't really need to be? Innovative or be innovating in order to succeed in the marketplace. What you need is just to be able to plug in and get out of the way. And in which case, if that's your thesis, it looks a lot more like the cloud providers of the world where you don't need like brand affinity really just comes down to  what your privacy and cyber security policies dictate.

Aaron: [01:06:52] Who gave me the best deal?

Phillip: [01:06:55] Yeah. Yeah.

Aaron: [01:06:55] I've heard the phrase "commerce is a commodity" so many times. I've said it so many times in the last five years. I think to the point taking it back to the quadrant. What that dichotomy that you just illustrated between technology providers who see commerce as plumbing and wiring, I think, are found on the quadrant. They are often those headless only, selling a set of unopinionated APIs, selling packaged units of functionality. PBCs. That's a vision of commerce that fits well into a large enterprise where, yeah, I mean, the transactional piece of commerce is fitting in alongside a very established DXP, a very established PIM, an ERP and order management system. And so I have a very narrow scope for the problem I'm trying to solve, so I want a tool that is like super scalable but super narrow in scope. Whereas other businesses I think are looking for more of an all in one. I want a commerce platform to be the beating hub of my business because it is my primary revenue stream and I need it to handle digital experience, personalization, content management, in addition to oh yeah, by the way, add to cart and checkout. So there's room for both.

Phillip: [01:08:13] Obviously there's room for both. That's what the Magic Quadrant is telling us. There is a resource that gives you one person's perspective, ex-analyst too, perspective. Brian Walker wrote up a very long, extremely, very long piece on his thoughts about the most recent Magic Quadrant. I think that it gives you one insight. So Brian's former Forrester, is that right?

Aaron: [01:08:42] Former Forrester. He was most recently Chief Strategy Officer at Bloomreach for the last several years, and has recently left to start his own consulting business. So props to Brian. He also makes a really mean cocktail, or so I hear.

Phillip: [01:08:55] Apparently. Or at least that's what the first 4500 words of the write up seems to suggest. What I love about his particular write up from Cocktails and Commerce, which is a Substack that he operates, where he writes about his views, his very opinionated views of this particular Magic Quadrant, some of which I think sparked a lot of debate amongst folks in those in the ecosystem, those who are in the know. I think it's worth a look if you want one person's pointed opinion and albeit he mentions multiple times like I think he throws some shade at Gartner because he's ex Forrester. So he has a little bit of the like collegiate allegiance, alma mater, you know, chip on his shoulder still, which I completely understand and get. And I love that he owns up to it. I think that there's just a really interesting inside baseball nature to how people view other competitors reports and how they might level critique against it. And the reason I think that's really interesting is because it shows that there's still a whole lot of passion left in this space, and people who care very deeply about the craft of the work. And so with that, I think it's worth taking a look at, and we'll link that up in the show notes. I really appreciate your time, Aaron. Thanks for joining us. We'll have to do this again.

Aaron: [01:10:15] Yeah. Phillip, thanks for having me. It was a blast.

Phillip: [01:10:17] And thank you all for listening to Future Commerce. You can find more episodes of this podcast on all Future Commerce properties at FutureCommerce.com. And if you want us to be in your inbox, you definitely do because we can see around the next corner for you, make you look real good at your job, and you can understand the world on a deeper scale. We can do that three days a week Monday, Wednesday, Friday. We help you see around the next corner. Get that in your inbox for totally free for the time being anyway. FutureCommerce.com/Subscribe. Thank you for listening to this episode. We'll see you next time.