Show 2: A New Paradigm: Streaming and Microservices

Podcast transcript:

Jim Scott:

Hi. My name's Jim Scott and I'm the host of Data Talks. Today's guest is Dean Wampler. Dean works for Lightbend and today, we are going to be discussing streams and microservices. They are pretty hot topics these days, and Dean is one of the foremost authorities in the area. I'd like to think that I am, as well, but well, that's why I have Dean on. So, Dean, I was wondering if you might be able to tell us a little bit about yourself. And when I say a little bit, I mean, very little.

Dean Wampler:

Yeah sure, Jim. So, I work at Lightbend, as you said. I've been running the team that built our fast-data platform, which is a distribution of open-source streaming technologies, like Spark and Flink and Akka Streams, Kafka Streams, we like Kafka as a data-backed plane. And so that's been our focus, is building this platform to help make it easier to deploy these things and then helping them actually do it when the time comes.

Jim Scott:

So, what is your educational background, just so people have a little bit of an idea of where you come from?

Dean Wampler:

Yeah, so I sort of ended up in this business, kind of in a circuitous way. I actually went through a physics, I've even got a PhD in Physics. But what I was doing for my doctorate was really computer programming. I was doing theoretical physics, isn't as impressive as it sounds. I was mostly writing programs, found that I enjoyed it and was pretty good at it, so I just made the jump after finishing my doctorate. I've been working in software ever since. This is a little over 20 years now, I guess. Actually, almost 30, really. But writing programs, everything from embedded systems to internet companies and more recently, like data centric technologies, start to dupe, and now more towards streaming and microservices.

Jim Scott:

Great. So, really, you're kind of like the epitome of the Big Bang Theory?

Dean Wampler:

Yeah, I guess. Yeah, just making a big mess and leaving a lot of carnage afterwards. That's kind of me, yeah.

Jim Scott:

So, one of the things that I hear a lot, out in the tech channels is people that are discovering microservices. They often bring up the topic of well, I remember something called service oriented architecture. Then, of course, I give them my response, but I was wondering, can you provide, in your words, a short explanation of what is really the difference between service oriented architecture, or SOA, and microservices?

Dean Wampler:

Yeah, one of the interesting things I think about our business in general, just to step back a little, is oftentimes we'll come up with some good ideas, but the first implementations will be absolutely terrible and people will run away and then come back with a different perspective later and come up with something better. So, I think a lot of the ideas were good about building things that are oriented toward solving particular problems people have that integrate into some common infrastructure to make it possible, but we just kind of messed up the way we did it before. And another example of this is CORBA, actually. You know, it was a fine idea to have a rich protocol for exchanging information, but it turned out doing it in binary was a [inaudible 00:03:17] idea, and doing it with text, which sounds like it's too stupid to possibly work, actually turned out to be a great idea and that's why we have the internet, basically.

Jim Scott:

Yeah, I totally agree with you. That analogy couldn't be more spot on, really. I hadn't come up with such an analogy for protocol for RPC, but it's really a major deal.

Dean Wampler:

Yeah. It's funny how we ... I think we all have this problem, we poo poo simple ideas either because they're not fancy enough to be interesting to us, or we think they can't possibly work, and yet often, that's where we have to start and then build up sophistication on top of it somehow, so ... I think that's true here with services that, what we think of as microservices today are still sort of a similar idea, but we've come up with good, practical ways to make it actually work. You know, very thin interfaces between things as much as possible, use of technology that's crucial like continuous delivery and continuous integration. Immutable infrastructure is a great idea, too. We try to avoid patching something, instead we deploy new things and then have a process of moving work to the new things without actually modifying existing stuff. So, all of these things, I think, are sort of pieces that make it possible.

Jim Scott:

Well, that's actually, I think, a very good segue into something that I just saw. It appears that Lightbend posted a blog I think today. You guys are going to be open sourcing some code for working with state in Kafka, and it's going to have an HTP interface on it.

Dean Wampler:

Yeah, it's actually, what we've done, it's ... So Kafka Streams has this ability to query its state in memory. And this is something a lot of the streaming tools, like Spark and Flank have added, as well. But, it was convenient to get at it, in part, because if that state was actually distributed over a bunch of nodes, you kind of had to query all of them. So, all we really did is write some tools that glue together capabilities that are already in Kafka Streams and just leveraged something we already had in house called Akka HTTP, so that you could more easily get all of the state at once without having to know where to ask for it.

Jim Scott:

Interesting. So, that will certainly help for those folks who want to move in that direction of having something an immutable model, correct? Being able to move services in and out?

Dean Wampler:

I think so. In this particular case, the idea of querying state is it eliminates the need to write the state to some persistent store. Then I query that instead, so I get rid of that complexity if it's ... I wouldn't necessarily use it for something ... Well, I don't know, I probably wouldn't use it for something where I needed to keep that state in a durable way. I would probably still have that process of writing it to storage, but if, let's say, I have a dashboard and I just want to see what my Kafka Streams are doing, then it'll make it easy for me to just query them periodically and find out.

Jim Scott:

Well, that's a really good point that you bring up, and it's one I don't really see a lot of people, say, paying attention to, is what are the ramifications to having that durability, that state? When I look at something like Apache Flink, and it has the ability to maintain its state or persistent state, in case of failures, things of that nature, what are people doing here? What's important?

Dean Wampler:

I think it's a dilemma, because if the process goes down and you don't have a backup, then you're going to lose that work. So, I think this is one ... You could ... This is possibly one way you could persisted, you might want to be more aggressive about proactively pushing to storage, periodically. There's also mechanisms, like check pointing and so forth, that are just designed to help you restart without data loss.

Jim Scott:

Interesting. When it comes to the streaming technology, is there one that you personally a bigger fan of than another?

Dean Wampler:

Yeah, so, this is a really interesting question. We deliberately took the approach of trying to be a Switzerland about stream engines. We couldn't support everything out there, there's hundreds ... Well, that's an exaggeration, there's maybe dozens that they claim to be streaming engines. But what we tried to do is decide on the projects ... You know, a small set of projects, we ended up with four, that had vibrant communities, had unique capabilities to kind of cover the full spectrum of what you need, and what we ended up with were sort of on the services side, where you have services running, and you submit work to them, that would be more like what you do with Spark and Flink, whereas if you just want to do microservices, where you have more control or need more control over everything that's going on, including how work is deployed, then you might want to do Akka Streams or Kafka Streams instead.

Jim Scott:

Interesting. So, if we digress for a moment, one of the things that I really like to try and pay attention to is what's going on out in social media land. And I have a number of people that I try to pay attention to on services like Twitter, you know, whatever your favorite social networking site is ... I'm curious, do you have a favorite person to follow, who you like to say, glean information from when it comes to topics like dub ops, data ops, streaming and microservices?

Dean Wampler:

Yeah, there's a couple I'll mention. One is actually related more to deep learning. I've been trying to keep up with that and there's a guy, I think he's in England, names Miles Brundage who has a Twitter feed called Brundage, but he posts links to the papers that have been posted to the archive, its arxiv.org site. So that one's been kind of interesting, you know it's kind of a firehouse and these are all technical papers, so there's no way you could read them all, but just to give people a sense of what's going on. I like him.

On the streaming side, one I've actually been following lately, the guys at Streamlio, Stream-l-i-o is how it's spelled, they come out of like Yahoo and Twitter, and they were involved at Twitter [inaudible 00:09:10] Heron, they're behind Apache Pulsar, I mean, these are kind emerging as alternatives for Kafka and I think they have a lot of interesting things to say about the pros and cons of Kafka versus other things like Apache Pulsar. So, even though we're kind of focused on Kafka at Lightbend, I think people should know about alternatives and make sure that it's the right fit in each tool you choose. So, that's another interesting one that I think is worth looking at, is their Twitter feed.

Jim Scott:

Yeah, and you know, myself, I've got some that I really like and I think I glean a lot of bits of useful information and the person who I would reference right now, his name is John Arundel. His Twitter handle is @bitfield, we'll post that after the podcast goes up, and he ends up saying a lot of very interesting things that I've personally lived through, I know others have lived through and so I find it insightful and kind of helps bring balance because you understand that it's not just you who's having these problems.

Dean Wampler:

Yeah.

Jim Scott:

So, it's the ... You know, I can self reflect and know that I had these problems, but when I see other people having these problems, it gives me a moment to be able to laugh and know that I'm not the only one. You know, helps preserve my sanity.

Dean Wampler:

Yeah, for sure. That's interesting, yeah, I'll have to check that out, too.

Jim Scott:

So, when we look at these scale out architectures, they're creating pretty massive changes throughout the industry. And one of the things that comes along is the concept of a data fabric. Still, that being a kind of a way to interact with your back-end storage systems through a variety of services, perhaps, microservices, being able to decouple these perhaps through event streams and this helps simplify or even hide the detailed implementations that are sitting behind the scenes, really helps deliver a pretty strategic agility for a business. Would you agree with that general notion of a data fabric?

Dean Wampler:

Yeah, I think the devil's always in the details. So, there's several ways you could incarnate that, that I think are very interesting. And one of them you see used a lot, for example, was Spark or tools like [Aluxigo 00:11:21], that are basically like a distributed cache, but it's sort of a primitive level, the same idea of a fabric. One of the extreme, I call it extreme views for lack of a better word in microservices, is that every microservice has its own persistence. So that, the idea being that if they're not decoupled at their persistence layer, then there's still a monolith in some sense. But I think a data fabric is a potential way to, how our vets are incarnated, it's a potential way of avoiding that problem, even if you have to have some fairly common databases behind a bunch of things. At least the data ... It's the usual, it's another level of obstraction or of indirection. It can be a very good one, especially if it hides details of the underlying, actual durable storage.

Jim Scott:

Right. And to your point to add on, the really important concept there, that I think some people ignore either by choice or by just sheer lack of knowledge, is separating out the notion of reads from writes with those systems and being able to get the true scalability. Otherwise, yeah, if you're mixing the reads and writes of these different services, you are effectively dealing with a monolith architecture. If two different microservices, that are completely isolated from each other, are reading and writing on their own to the same storage, they are effectively, even though deployed as microservices, they're just one service. You just deployed them as independent services, right?

Dean Wampler:

Yeah. Yeah, that command query responsibility separation is a really interesting, and not really a new idea, but it's been around a while, but it's really emerged as something very, very important for this kind of resilience and scalability and if you can scale ... Like if you have massive reads but relatively few writes, then you don't want to have a compromise solution for just as an example of where that comes to play.

Jim Scott:

So, the real question that I have for you when it comes something like a data fabric, what are the specific technologies that engineers and architects should consider looking at when building their myriad of microservices to support this type of a data fabric that they'd implement for their solutions?

Dean Wampler:

It occurred to me as you were talking a second ago, that one of the things I would want to do first is make sure I've figured out these kind of competing requirements between performance versus like flexibility of implementation and resilience and so forth. What I'm getting at is, sometimes you really do need just a direct pipe between some data store and data sync for maximum performance. We give up some flexibility, scalability perhaps, in other ways. So, I would want to figure those things out first. And then, I would certainly look at relatively low overhead and low sophistication solutions. What I'm getting at are things like these distribute caches that they solve a core problem, but don't try to be everything. So, I would hesitate to do something like introduce some [inaudible 00:14:20] different data stores, maybe one completely different store for reading, one particularly different store for writing, just because you like the features they have. Then you've added complexity in terms of operations and then sort of the mind share you have to acquire to know how to use them well. But if you can find a middle ground or something lightweight, like a caching layer lets you use one persistence engine, then I think that's a win, for example.

Jim Scott:

That's a good point. And it does remind me of another topic that, we'll say is maybe political in some people's minds, but it's one that I'm rather passionate about. I know you and I have talked about this in the past and I have this conversation a lot with people. Where do you fall of the topic of consistency? Eventual or just give it to me and make it consistent?

Dean Wampler:

You know, this is a really interesting question. I have to say that I really love ... I think the theory ... It's sort of a great compromise between theory and practice. The theory of eventual consistency is really very enticing and there's a beauty about it. The thing you read about experiences at places like Google, where they found that it was really, really hard for developers to write reliable apps when they had to think about this problem and if eventual consistency and so they've tried really, really hard to make things, to emphasize instead availability so that you don't have to give up consistency in some atomic low. I think if you think about eventual consistency as really a compromise, that it's not necessarily something great on its own, but it's a compromise solution, then you're probably better off. And if you can avoid it, great, but if you can't, then know that it's gonna be a harder thing for you to work with and if you could just assume the world is perfect.

Jim Scott:

Yeah, and I think, the fundamental problem that I have with it is, there's been this misinformation around eventual consistency that you're getting something amazing and at the end of the day, it is possible for that system, like a Cassandra, with eventual consistency to become unavailable. When the trade-off is supposed to be consistency. So, if you suddenly have to go from reasoning about not only about consistency, but now reasoning about availability, you're literally reasoning about the two things that you were debating before you chose your data store before you should have made the decision. And if your system can become unavailable, then you shouldn't ever reason about consistency because your point is exactly that. When you make a software engineer or software architect have to think through all of the possible paths that data can take, networked apologies and things of that nature when it comes to maintaining consistency, it is a nightmare to manage.

Dean Wampler:

I think the lesson here really is that, figure out when it's strategically necessary to go with one versus the other. So, just as a made up example, maybe I can live with consistency in one data center, but maybe I have two for complete fail-over. In that case, you have to live with some degree of eventual consistency just because of the time of flight for data replication. So, in that sense ... But at least you sort of isolated the problem to something that relatively constrained and hopefully you can figure out the right solution and the smart guys can focus on that while everybody else works about their applications and lives in a more perfect world, so to speak.

Jim Scott:

Exactly. So, when it comes to the technologies for implementing these microservices, streaming applications, we want to get to a "I'm going to build my components, I'm going to deploy them, manage them." What are some of the frameworks out there that you're interested in, you keep your eye on, that you recommend to people to use to build their software?

Dean Wampler:

Yeah, obviously, coming from Lightbend, we're partisans for our own reactive platform, which is Akka and Play. Scholar's an option, but you can use Java as well. We've thought a lot about this, even before we started thinking about data, about how to build what we call reactive services, and reactive is designed be kind of like a high level manifesto in the same way that Agile is, where there's a few core principles you want to have, like event-driven and scalable and resilient against failure and so forth. Those are the sort of things that we try to live by and so that's why when we implement stuff, we really like using these tools that have, like for example, Akka, I think was started in 2009 and so it's had like 8 years almost now to really bake and just become quite robust and quite sophisticated in what it can do, but going back to what we were just saying a minute ago, it hides a lot of the sophistication of concurrency and distribution behind the scenes so you can focus on the main [inaudible 00:19:07].

Then, on top of that, we're very much embracing the Kubernetes model as well as, like anything, it's probably not the most perfect resource manager, scheduler kind of tool, but it has a lot of momentum now. So, that's what we're recommending that clients use for building microservices. Our fast-data platform is actually built on top of Mesos, which is, that's been an interesting choice, too, but it's actually turned out to be a good ... Just even with the Kubernetes craze because there's a good, solid commercial distribution Mesos from Mesosphere called DC/OS that has solved a lot of the problems that we would have to solve ourselves. And now, they're embracing Kubernetes, too, so you'll have sort of the best of both worlds.

Jim Scott:

So, Kubernetes is certainly a very, very, very hot topic right now. And it has certainly taken the throne as the orchestration layer of choice. What are some of the shortcomings that you personally have seen in Kubernetes that, whether it's being worked on or not, there are things that people are struggling with or that they should look out for?

Dean Wampler:

Yeah, I haven't had a lot of time to dive into it myself, yet. Though, I'm not really an expert on this. The things that I've heard mentioned would be really sophisticated ways of managing stateful applications versus stateless, that's sort of relatively new capabilities are being built in. There's a whole lot of stuff that you would get from a mature product, like you guys, for example, for managing things like data governance, security, multi-tenancy, these are all things that are still relatively raw and new in the Kubernetes world. Again, things that everyone is working on, to one degree or another, but enterprises as we all know, they can't necessarily be on the bleeding edge, they have very real, sophisticated requirements that have to be met, so that's why there's a market for people like us, who are selling stuff that may not be the sexiest thing right now, but it's solving the core problems that people have to solve.

Jim Scott:

Since you brought it up, a point of this podcast is not for me to go spouting off about every great thing that MapR does, but that is definitely one of the things that MapR has helped to solve, which is enabling statefulness from within those containers by merging MapR as a persistence platform. Whether it be for the state coming out of applications like log files or through the database or for the event stream. So, it's something that does pretty easily plug into those containers and make it so that when you leverage it with something like Kubernetes, it gives you some pretty strong capabilities.

My one thing that I've seen with Kubernetes, though, is the comparison of "Well, can I run Spark on Kubernetes?"

Dean Wampler:

Right.

Jim Scott:

And I think when that finally shows up, that will turn some of those other arbitrary resource negotiators that are out there on their side and cause them to have to rethink whether or not they belong.

Dean Wampler:

Yeah, I agree with you. That's going to be a turning point. I know there's work being done on Kafka and HDFS, as well. For a typical data environment, they're going to have to have all of those.

Jim Scott:

So, I have a little thing I like to do during these podcasts called the Tip of the Day. Now, the Tip of the Day, in this case, happens to be something that I was reading, just today. There was an article on testing microservices, written by a lady named Cindy Sridharan. Her Twitter handle is @copyconstruct. And I would suggest anyone working with highly scalable architectures, with decoupled message-driven ambitions and microservices to consider giving this article a read and we'll have this posted with the podcast. Her article was posted on Medium and the actual tip to take away from this is she references something in her article from an extract from a Google blog post entitled "Testing On The Toilet", which that's hilarious that's the name of their article that they had.

But in this little extract that she points out, and this is absolutely a critical topic when it comes to testing, is they call out and talk about a few different types of testing. Number one is using stubs. A stub, which effectively has no logic, it only returns what you tell it to. And then there's mocks and a mock has expectations about the way it should be called and it tests and verifies that, if it's not called that way, it fails. And then there's fakes and a fake doesn't use any mocking framework. It's a lightweight implementation of an API. It's intended to look like a real implementation, but it is not a production suitable implementation.

So, these are all things I've used in the past. I, honestly, feel like they don't get talked about enough. And now, Dean, I know that you and I both put a considerable value when it comes to testing. Especially doing testing before you go to production. So, I feel like with microservices, since they're such a hot topic, I just feel like there's a huge gap out there, when it comes to really, truly understanding how to test microservices. This article that I'm referencing, it has a lot of information, but ... What suggestions do you have for people, when it comes to actually testing these architectures?

Dean Wampler:

Yeah, this is obviously a huge topic and it doesn't just end, of course, when you finish your tests because your production environment invariably is going to be different and something could come up that you didn't anticipate that has an unusual side effect. Like this whole Intel bug recently. That's really kind of a big side effect that you wouldn't necessarily know to expect if you were testing in the usual ways. I think one of the things people should do is, obviously, you have to design for testing so that it's easy to insert these mocks or fakes or stubs or whatever component you're using.

The other thing that I think that I see people not being able to do a lot, just in customer support, not to complain about our customers, I love our customers, but one of the weaknesses I sometimes see is people don't really know how debug performance problems. And I think it actually starts at testing with [inaudible 00:25:18] to know how to substitute in test harnesses they can exercise some things, so you can not only look for flaws that are ... Logic flaws are the usual bugs that we think of, but also what's the performance profile? And how do I characterize that so that when I put it together with the real system, that I'll know where bottlenecks might be, where I have to be able to scale.

So, this goes back to what we were saying earlier about the kind of design trade offs you make. One of them might be, I need a lot of flexibility to scale, this component up and down in this chain of components, in this graph of things because this is where I know the bottlenecks are from my exhaustive testing. And so, it's worth putting the energy here to be able to scale that thing up and down, even if everything else is relatively static. So, I think there's a lot of work that people have to invest in, in building these kind of harnesses to do this sort of test, so that they can not only squeeze up all of the bugs, but kind of anticipate as much as possible, the real-world scenarios that they're going to encounter.

But I think that the flip-side is that once you're in production, systems are gonna be evolving, I think, to be much more dynamic and responding to real-world conditions. One of the interesting things that has just come out of Google recently is a paper on using neural networks to do heuristic algorithms in ways that we normally would have done with some sort of heuristic algorithm, but now they just sort of watch what's gone and then optimize the behavior in real-time. And it's kind of a logical extension of the way the JVM, for a long time, has been able to observe your code running and then know where the hot spots are and then optimize those into real actual CPU instructions.

So, these kind of things, I think, will also be kind of the flip side of the coin of systems that are designed to learn from the real world [inaudible 00:27:08] in which their running and to self-optimize in ways that keep them healthy much longer than they would otherwise.

Jim Scott:

Yeah, it definitely is a huge concern for me when people are looking at leveraging these highly scalable distributed systems and not knowing exactly how to be able to run integration tests or unit tests appropriately with those to understand what the dynamics are of how data is moving across the network. Right, how is it actually going to scale, how can you actually test this to know and have an expectation when you make it to production what's going to happen?

Dean Wampler:

Yeah, exactly. It's circling back to the original question about service oriented architectures. I mean, these kinds of requirements for this ... How something has to live within this large environment, these are the sort of things we didn't really understand when SOA was first being articulated. So, we would do stupid things, like allow unconstrained mutability and so forth, which made it impossible to really know what was going on internally, but now, we're getting to these ways of thinking about these problems that enable this kind of plug and play architecture.

Jim Scott:

So, we're getting close to the end here and when we get close to the end, one of the things we like to do is give our guest the ability to offer up what they think are some action items to our audience, that they can take back and make themselves more successful when it comes to their implementation. So, what are some things you think our listeners could do over the next three to six months to really improve their experience with streaming systems, event streaming architectures, decoupled message-driven architectures, whatever term or phrase people want to use and being able to build and be successful with this microservices construct?

Dean Wampler:

Yeah, that's a good question. One thing I would say is that it's a fast-moving section of our industry now. So, you really have to allocate some of your precious time to actually learning what's going on, keeping up with new developments and understanding the core principles. A good place to start would be, like Tyler Akidau from Google, wrote a couple great blog posts over a year ago for O'Reilly, describing the semantics of stream that you really have to be thinking about if you are going to do in a sophisticated way. I think this year, a book that he's co-writing, will come out with a lot of this wisdom.

So, those are the sort of things I think you have to carve out time [inaudible 00:29:34] understand so that you can make more informed choices about how you should build stuff, what tools you should pick, how to trade off different constraints. And then a lot of it is just learning, going back to what we said a minute ago, how am I debug this stuff, how am I going to test it before I put it into production, how am I going to monitor things, and how am I going to build things so that they're more resilient and more flexible. And I think that's ... It's a tough challenge, a tall order. But, in principle, the results can be very rewarding for the company in terms of competitive advantage and all that stuff, so I think it's worth expending the effort, but it will require some investment in learning and experimenting.

Jim Scott:

Yeah, I agree with that and your mention of Tyler Akidau's posts on O'Reilly, I believe they're titled something like Streaming 101 and Streaming 102.

Dean Wampler:

Yes, yes, you're right.

Jim Scott:

And they are fantastic articles and I would definitely suggest them to anybody out there to go take a look. I think, for me though, kind of the if I boil it down in summary, what I like most about those articles is that they're basically saying, look, people used to be accustomed to doing things in batch and so that's how they've started moving into this new technology stream. However, if you think about things in a streaming-first model, suddenly, you realize that everything is actually streaming anyways. [inaudible 00:30:54] going to batch actually just slowed you down because everything was already coming in in a stream so if you can just operate on it, in its natural form, well, you're going to be better off.

And it flies directly in the face of the lamb to architecture, which is hey let's combine these two things and build a system on it, which introduces a significant amount of complexity.

Dean Wampler:

Yes, you're absolutely right. It really is a see change in thinking about these things. But that is important. Yeah. You have to be willing to go in with an open mind and realize that your assumptions might not survive at the other end.

Jim Scott:

And then, one finally question for you is what can our listeners do to help enable their architects, engineers and data scientists to work together more effectively? Because data ops is really about bringing those together and saying that data is just as important as the software that was used to create it. The data models matter as much as anything else. How do they work together more effectively?

Dean Wampler:

Yeah, this has been tough and there's solutions, everything from things like using tools that let you share completely different kinds of code [inaudible 00:32:04], I'm thinking about Spark, in particular, where you could have your infrastructure and your regular pipelines written in Spark with Scholar or Java, but to actually deploy machine learning models written in Python or something. But I think a lot of it comes down to building a culture where people are, they sort of treat each other as customers, in a way, so in the same way that you would spend time with your customer to understand their requirements and to sort of learn to anticipate their needs ahead of time. You have to build that same kind of culture between these really direct teams.

In a way, it's not a new problem, of course. I mean, UI versus back end developers have dealing with this for decades, but it's the same kind of principle. How do we both bridge these gaps technically, but also conceptually in the way we think about things? So, that's what I like to do is try to get the teams talking as much as possible and then understanding each other's concerns.

Jim Scott:

Yeah, and I think it is one of those things where it sounds so simple on the surface and yet, at the end of the day, it is a very complicated, tough thing because of the different communication styles that people have, the cultures that exist within these different companies, the walls that are put between the different groups for how they operate. So, it is not a short, easy task.

Dean Wampler:

Yes, for sure. And you know, like anything, it takes investment to make it actually work and a lot of people are figuring out how to do that.

Jim Scott:

Well, that's all the questions we have for today. And for all of us here at Data Talks, I'd like to thank you for listening. I'd like to thank Dean Wampler for being a guest with us here and this was ... Honestly, this was a fantastic conversation. There's a lot of good take-aways for myself. Hopefully, our listeners will find it to be very fruitful. And there's two additional resources I'd like to make sure people know. Dean wrote a book entitled Fast Data Architectures for Streaming Applications and it's available on the Lightbend.com website. And I'd also like to recommend a book that I wrote to help you learn more about how you can better leverage streaming microservices, containers and the Cloud, titled A Practical Guide To Microservices and Containers, which can be found at mapr.com/ebooks.

So, be sure to tell all of your friends, maybe even your grandparents, at how fantastic this Data Talks podcast was. We look forward to speaking to you again in the future.

Subscribe to the Podcast

Be the first to hear our newest interviews and other DataOps topics

Subscribe Now

Data talks - A Podcast to Help You Drive a DataOps Culture

Subscribe