I think i’m getting old... was telling coworkers that their fancy micro services existed 15 years ago in the form of EJB and service bus was a thing. And look, here it goes again
CORBA and the object mapping model (similar to the issues of ORMs) is what led to the dreaded ClassCastException problems that required every service to re-deploy at once which was completely broken and indicative of the architects being engineers that hardly ever touched day to day production. The transition off of syncing object representation in a monolith and God Service Schema / Registry was necessary but by that point we had moved on to SOAP and screwed that part up once again by needing to re-deploy JARs yet again constantly and out of sync and also being unable to separately deploy and route components off which led to either more ditch digging via OSGI or by going to REST APIs... or if you didn’t get here you used earlier incarnations of object and message serializations with protocol buffers. Only with these painful lessons have we as an industry learned how to actually decouple our services’ business boundaries well enough to call them decoupled, scalable services.
Did we learn the lesson? Protobufs are getting traction, and some rising star companies started using them for data related purposes. The problem they are facing - change in the protobuf is not automatically reflected in you data schema (not like with avro), you have to recompile and redeploy pieces of your stack + orchestrate deployment so data is not lost in processing. It can be solved easily with avro, but “we want to use protos, Bla Bla Bla”
Thrift, ProtoBufs, and Avro solve the messaging compatibility layer problem at least (optional fields in a schema are all you need 99%+ of the time) and so far nothing out there has managed to solve any data schema incompatibility problem automatically because that’s pretty much intractable, but providing tools that make finding problems earlier certainly helps with the reality of SOA in its various incarnations.
Agreed completely, programming patterns are just like anything else, timing is everything. I thought that ESB pattern was cool in the early 00s, today that pattern has touched off and evolved a robust ecosystem. It’s totally fair to rebrand it as a descendent of ESB, it’s fair to point out the lineage, but ecosystem evolution + pattern refinement is a distinct thing to get excited about.
After reading it I can't understand how EventBridge is different to just using SQS or any old messaging service? It just sounds like a message system layered on a messaging system. Can anyone elaborate why this is significant?
It has built-in integrations so you don't have to standup a service or API Gateway to receive webhooks from their list of partners. Their initial list of partners is [1]:
- Datadog
- OneLogin
- Pagerduty
- Saviynt
- Segment
- SignalFx
- SugarCRM
- Symantec
- Whispir
- Zendesk
and of course that'll grow over time - anyone that produces events can become a partner.
It's one of those things that you don't know you need ... until you get half way to re-inventing it yourself and then you realise you needed it. Wrt this, probably the more relevant part is all the end point integrations:
Camel was libraries that codified the the Enterprise Integration Patterns (https://www.enterpriseintegrationpatterns.com/ - channels, messages, service activators, etc...) that allowed you to code against the abstractions rather than the underlying implementations (JMS w/ topics & queues, AMQP w/ exchanges, etc..)
Spring offered a competing set of libs with Spring Integration.
Those we mostly just libraries, though, so you have something running under them for distribution (like a message broker or service bus or, these days, a cloud managed service).
> so you have something running under them for distribution (like a message broker or service bus or, these days, a cloud managed service).
Actually Apache Camel doesn't really care if you have a message broker or not. You can set it up without one and still use half its functionality. (eg: connect a file arriving in a dropbox account to an outbound SMTP etc).
This looks similar to Azure Event Grid, which I'm more familiar with. If that's true, the value prop is that one event handler can be webhook based while another handler can be queue based, and you can add and remove these different kinds of handlers without making any producer-side changes. It's quite nice
Not a ton of outside integrations, but this library creates and maintains an esb using sqs and sns with just a few annotations in java. Uses json as the message and maps to pojo in the application.
How is this "SNS and SQS rolled into one"? What's the SQS-like part in this? This seems very similar to SNS, with some additional flexibility around event filtering, and the added bonus of various SaaS integrations that can replace traditional webhooks.
EventBridge is a PubSub system with different tradeoffs and operational constraints and behaviors.
The cool new features are: (1) It allows serverless dynamic routing based on message content which auto-scales to match usage. (2) It supports SaaS partner events. (3) It supports publishing events based on a scheduled configuration, which can be used to trigger cron jobs or other system processes.
So it is just a messaging system, but one that might be more appropriate to certain use cases over using SQS or SNS or Kinesis, etc.
The article thinks that the ability to have SaaS partner publish events will be a huge deal and is a killer feature of EventBridge.
it looks like it only replaces SNS no? As some of the examples are Lambdas and SQS queues that you can publish to. We have a small Go service that basically maps endpoints to SNS topics which then publishes events to SQS or Lambda or some other internal endpoint.
Don't see how this is much different other than provided integrations with popular services out of the box.
Amazon is a logistics company, not a cloud company. Their product strategy is spray and pray. "You want a DB? We have 10!" GC has fewer, more stable, high quality offerings. Amazon is like their store "Want shoes? We got millions." Google is like their site "You want search, we do search."
As people say in sports when someone is trash talking when they are losing: “Scoreboard”
The AWS approach (and Azure) is clearly working. And the reality is that people want multiple approaches, especially enterprises, which are typically migrating applications to the cloud and can’t just rewrite everything for Google’s random proprietary platform.
Also, I’ll note that GCP is pretty aggressive about EOLing platforms. Good luck with that in the enterprise market.
We have a very small Golang service that exposes some endpoints and basically just forwards incoming events to SNS which then publishes to subscribers like SQS/Lambda/whatever. This small service basically never fails as it's only dependency is SNS. I don't really see how this EventBridge is much more than that?
I have a very similar setup what you have. I think what event bridge offers, is the ability to integrate with anything which can reproduces event. They are trying to expand their base beyond AWS ecosystem. Datadog, Zendesk, OneLogin are great services which could be helpful, to begin with. Still time will tell how popular it gets.
> EventBridge was formerly called Amazon CloudWatch Events. It includes new features that enable you to receive events from SaaS partners and your own applications. Existing CloudWatch Events users can access their existing default bus, rules, and events in the new EventBridge console and in the CloudWatch Events console. EventBridge uses the same CloudWatch Events API, so all of your existing CloudWatch Events API usage remains the same.
So EventBridge is CloudWatch Events. The difference is that now you can publish your own events and specify your own routing rules for them. As well as supporting 3rd party events from certain SaaS partners.
It is limited to 400 events per second, 5 targets per rule, and 100 rule per account. Consuming events is throttled at 750 per second. Events take about half a second to reach targets. Some of these limits can be increased by asking for higher limits. AWS published events, so existing CloudWatch Events don't count towards these and are unlimited.
What about SNS?
> Amazon SNS is recommended when you want to build an application that reacts to high throughput or low latency messages published by other applications or microservices (as Amazon SNS provides nearly unlimited throughput), or for applications that need very high fan-out (thousands or millions of endpoints)
> At launch, Amazon EventBridge is has limited throughput (see Service Limits) which can be increased upon request
> Amazon EventBridge is the only event-based service that integrates directly with third-party SaaS partners
> Amazon EventBridge uses a defined JSON-based structure for events, and allows you to create rules that are applied across the entire event body to select events to forward to a target
Also, the targets are not exactly same yet. For example, only SNS can publish to Mobile SMS, Email, or HTTP/S endpoints for now.
And SNS allows any type of payloads, while EventBridge forces you into JSON.
Finally, the reason this specifies serverless is because of the dynamic content based routing using configurable rules. This means you don't need to manage your own routing. So say you want to pair it with Lambda, you don't waste Lambdas starting then realizing the event isn't relevant for them, you can just setup rules for that. So it provides a serverless routing mechanism that auto-scales and is decoupled from producer and consumer. That's pretty cool!
I would really like to see how this compares to Firebase. It sounds really similar in that you subscribe to events and then AWS or Google handles the difficult tasks of routing to the right places.
“It re-establishes how the greater ecosystem of software integrates with your business logic, in addition to defining an excellent standard for your own services to leverage.”
It's PubSub with dynamic message routing, but where each AWS account will also now be able to make themselves a consumer of events published by 3rd party SaaS. Say you want to start a Lambda whenever a Zendesk ticket is opened, you can now do that without having to setup any hosts or write any code.
I think i’m getting old... was telling coworkers that their fancy micro services existed 15 years ago in the form of EJB and service bus was a thing. And look, here it goes again