There's API gateway products (Apigee, Kong, etc). Load balancers and proxies of various types. Caching and CDN products. More niche stuff like bot blocking, and this attempt to bundle control and statistics.
It would be nice if some sort of standard pattern emerged, where something was the main orchestrator. At the moment, you can end up with suboptimal stuff. Like a CDN that routes to a cloud API gateway that then routes to a (not geographically close) load balancer, that then hits the actual service.
I'm surprised that Cloudflare, Akamai, and the like haven't offered all of these things at the edge. Some things are service to service, but a fair amount is client to service...putting this stuff closer would help.
Imagine you've got 50 microservices and you're using RPC to communicate. You're going to want global rate limiting and circuit breaking behavior to insure resilience, particularly as your topology gets deeper.
There is a use case for extending Istio to the Edge, which is why we wrote Ambassador (it's an API Gateway built on Envoy), and we just released an Istio integration (http://www.getambassador.io/user-guide/with-istio.html).
Full disclosure: Work on Ambassador
https://apigee.com/about/blog/digital-business/simplifying-m...
We're building something a lot like that! https://fly.io
Currently, we support AWS Lambda, Heroku, and self-hosted backends. Bringing things to the edge can give some major benefits; a smart proxy, too, can give developers a lot of power with minimal effort.
Redhat: https://blog.openshift.com/red-hat-istio-launch/
Pivotal: https://content.pivotal.io/blog/pivotal-and-istio-advancing-...
More info is available in the overview doc here: https://istio.io/docs/concepts/what-is-istio/overview.html
What's more, these things are being constantly deployed to a wide variety of environments. Some may be on cloud VMs (or a dynamically scaled cluster of VMs), some on bare metal, some in orchestrators like Kubernetes. Some will run on networks that the organization maintains, some may be maintained by a DC or cloud provider.
Historically the answer to securing this communication has been to use L3 network segmentation with strict rules to decide who can send packets to who. But, particularly in an increasingly heterogenous and dynamic environment it's very difficult to do this reliably and quickly. Networks are also a pretty crude authorization system - it implies that just because you can reach an endpoint that you are authorized to use it, which isn't necessarily true in practice. Some of the other benefits of Istio - like system-wide circuit breaking and flow control are also difficult to do purely at the network layer.
If you're interested in this, I'd encourage you to check out https://spiffe.io/about which has some more detailed thoughts on the limitations of the L3 micro-segmentation approach and how it can be solved.
Securing an endpoint without requiring changes within the endpoint has been done for some time - Whale Communications, which became Unified Access Gateway, F5 Big IP, IBM DataPower... They are called web application firewalls, and unless I'm missing something Istio is no more than that, but targeted at micro services.
Project Calico Network Policy Engine: https://www.projectcalico.org/welcoming-istio-to-the-kuberne...
WeaveWorks: https://www.weave.works/docs/tutorials/istio/istio/
Istio website: https://istio.io/
The problem with SOA-in-practice was that everything flowed through a monolithic ESB as both client and origin server, that needed to have omniscient knowledge of every route, transformation, etc., and was often a single administrative bottleneck, fault domain, etc. Some SOA frameworks had service mesh patterns where you could deploy decentralized engines with your services, but without cloud IaaS/PaaS circa 2006-2007, there was no way to maintain/deploy/upgrade these policy agents without a heavy operational burden.
In sum: CORBA, COM+ or SOAP/HTTP were about mostly-centralized approaches to distributed services, REST was about extreme decentralized evolution over decades, most are looking for something akin to a dial where they can have something a bit more controlled than dozens of independent gRPC/HTTP/Rabbit/Kafka producers-consumers but not stupid like the SOAP/HTTP days.
Modern cloud native service mesh approaches like this Istio thing (NetflixOSS Zuul+Eureka+Ribbon or Linkerd are alternatives) are just decentralized gateways and proxies, possibly with a console/management appliance that makes it easy to propagate changes out across a subset of your microservices. This has the benefit of allowing you to default to decentralized freedom for your various microservices but for areas where you want administrative control over policy over a set of them , you don't have to go in and tweak 15 different configs.
NetflixOSS really pioneered this pattern. Netflix managed to use things like Cassandra and Zuul hot-deploy filters as the means to updated routing/health/balancing configs across their fleet of microservices. There are alternative ways to handle this of course - Hashicorp's Consul piggybacks DNS and expects your client to figure things out via their REST API or DNS queries. There are also things like RabbitMQ or a REST-polling mechanism to propagate config changes perhaps, as not everyone wants Cassandra. New frameworks like Istio or Linkerd are further alternatives. We're spoiled for choice, better or worse..
Besides Netflix, I'd put Twitter as an early pioneer, with their work on Finagle. Both of these companies, for better or worse, took a library-centric approach (Eureka/Hystrix/etc or the Finagle lib). This limited their applicability to the JVM.
The sidecar model that AirBnb pioneered with SmartStack, later adopted by Yelp and others was the cheapest way to get non-Java langs to have similar resilience/observability semantics. And now given the popularity of polyglot architectures, it's probably should be the default choice for companies adopting microservices.
As long as proxies remain transparent to services, I see no problem. It becomes a problem, when proxies are getting smarter in terms of providing cross cutting features, like routing on payload level (not speaking of message headers) or do authentication and authorization on a per-resource level. That puts constraints on how services are built in this particular environment.
But I see the logic behind approaches, developed by Netflix, and now Istio. If you have a lot of services, orchestration and more central communication management is probably a good way to govern, if the constraints (described above) are accepted and services still have the ability to opt out and pursue a different strategy.
The old SOA world was driven by governance. This was a result of the general engineering methodologies and mindsets of this time. Still, API Gateways / smarter proxies / etc. could bring that back...
From a developer perspective, service-oriented just means that you're offering/accessing functionality via a well-defined app-specific network protocol interface with a standard taxonomy/representation of cross-cutting concerns such as auth, transactions/compensations, message synchronicity and QoS semantics (eg. request/response, at-least once delivery etc.), most of which define the shape of your service implementation code fundamentally. For example, if you're operating under the assumption that no distributed transactions are available, you'll have to fold the necessary logic for restarting and state management into your application code.
Take a look at AirBnB's smartstack, Yelp, Lyft's Envoy mesh. These are all polyglot applications, where the communication aspects are abstracted out into sidecars.
If the smarts is just a state machine, and the rules are abstracted at the right level, it would not be that hard to support.
"Thick" smartpoints can save on operating cost (infrastructure, personnel, complexity, level of expertise) when your system is "simple enough"
At massive scale, from an economies of scale (all things considered), it may make better sense to go with "Thin" smart endpoints.
The migration between the 2 models is where the art of the design comes in play :) It's an art because it is a decision process involving many players with conflicting and competing goals, decisions, time frames, motivations etc.
It's often not just a "technical" decision
app|agent <-> ... <-> agent|app
Fundamentally, it is the same problem space.
Does the apple pie taste differently with the pie in the apple vs apple in the pie? Maybe in some cases and maybe not in other cases
IBM: https://developer.ibm.com/dwblog/2017/istio/
Redhat: https://blog.openshift.com/red-hat-istio-launch/
Pivotal: https://content.pivotal.io/blog/pivotal-and-istio-advancing-...
Linkerd is great technology but it is restricted to traffic management only. Istio provides a complete mesh that incorporates authentication and policy enforcement, in addition to traffic management and telemetry.
The Istio Auth subsystem provides certificate management and we are working on extending it to support authorization primitives as well.
The telemetry model is also different. Rather than having direct integrations with different metrics backends, we normalize metrics and pipe them through a single engine that can then re-route to any metrics backends (or multiple).
In contrast to Envoy, linkerd provides a minimalist configuration language, and explicitly does not support hot reloads, relying instead on dynamic provisioning and service abstractions.
Disclaimer: I work on Istio (on Mixer).
Our deployment model is a bit more transparent. Traffic gets transparently routed via Envoy, without using HTTP_PROXY or direct addressing of sidecars. This implies zero change to application code.
Secondly, Istio brings two more things to the table: policy enforcement (rate limits, ACLs, etc.), and authentication/authorization. Istio enables mutual TLS auth between services with automatic certificate management. The policy plane is extensible, where you can plug in adapters or specific policy implementations.
Disclaimer: I work on Istio.
Istio and Linkerd are both very young and the adoption will be with those that really need their unique features. Most companies, including startups, in my experience are using roll-your-own routing/traffic mgt via Consul , or using NetflixOSS' stuff today and probably through 2018. Enterprises are buying API Gateways and the like too - IBM API Connect, MuleSoft, Apigee Edge, etc.
As for K8S vs Swarm, I rarely see Swarm in the wild, really. From my vantage point, the battle has been Kubernetes vs. roll-your-own-Docker vs. Mesos in startups using the pure open source. For enterprises paying a vendor, it's been IBM Bluemix vs. Pivotal Cloud Foundry vs. OpenShift, as they've been making more money out of all these container vendors combined (I work for Pivotal) - probably around $300m+ annually across the vendors. Some just use their native cloud's platform, like AWS container engine, etc.
The main X-factor will be serverless frameworks eventually taking over, possibly rendering today's battles over runtimes somewhat moot.
> a production system spanning 10,000+ VMs handling 100+ microservices.
Istio (or rather, Envoy) acts as a plain HTTP proxy, meaning a client can just respect the standard http_proxy environment variable (which most client libraries do), meaning a client can just do HTTP and doesn't even need to know about the proxy. Even "curl" will work with Istio. And it's even simpler on the server end; all a server needs to do work is to accept HTTP connections.
Last I heard, Istio aims to go even further by supporting the proxying of any networking protocol (Layer 4).
Maybe I should have blogged about it :)
Anyway, although all the design principles are the same: 1) Transparent and seamless to application code, 2) Control communication end points, the MQ constructs and languages are not as accessible and readily available and visible as HTTP and RESTful.
A MQ can be temperamental and hard to operate. On the other hand, even middle-schoolers can master nginx, haproxy, write their own, etc with a bit of guidance :)
Popularity and rate of adoption depends on availability to the masses and the ease with which the masses can learn to use. If it takes to long (say, 15 minutes) to bring a framework up, hordes of engineers will find a massive number of reasons to roll their own ...
I can't see a reason why one would use gRPC/thrift and then having to build flow control, delivery guarantees and other messaging features yourself.
But message queues are usually an added complexity for realtime inter-service messaging.
But it doesn’t help with higher-level problems, such as L7 metrics, rate limiting and circuit breaking.
This is where Istio comes in.
Disclaimer: I work on Istio.
https://apigee.com/about/blog/digital-business/simplifying-m...
Disclaimer: I work on Istio
1. https://groups.google.com/forum/m/#!msg/kubernetes-sig-netwo...
There is actually a lack of it, there is only connotative evidence here in one comment that refers to "hey look linkerd does this at scale" that sounds nice, except that is not necessarily the point or the case.
How is having side-loaded proxies _better_ or more beneficial than having it separate? It doesn't appear to be discussed or mentioned in this thread, all of the arguments made for doing so are equally applicable to the separation of the proxy.
No option for OAuth2 or JWT? Maybe I'm not understanding the problem Istio solves vs. Envoy
Disclaimer: I work on Istio
Disclaimer: I work on Istio
Are those numbers right? Wouldn't it be the other way around realistically?
You should be using service discovery and deploying multiple instances.
[1]: https://readon.ly/post/2017-05-25-deploy-istio-to-azure-cont...
Okay: I immediately deeply, profoundly, bitterly hate and despise your announcement and for just one, really simple, dirt simple, but totally unforgivable reason: What the heck is a "microservice"? You never said.
That word, microservice, is not in a standard English dictionary, so you are writing undefined jargon, gibberish, junk and not English. You are insulting me and even worse yourself.
Instead, write English. Get rid of the undefined jargon.
Got it?
This was a difficult lesson?
> the biggest impact
Until you learn to communicate at, say, the late elementary grade school level, e.g., learn to write English, the impact promises to be minimal.
Microservices are services that are generally containerized and are easily distributable through some form of a network to be easily replaceable parts. These services are defined by a specification where they do a single task, expose some endpoint or API and are composable with other microservices.
There is a need for these services to talk to each other, and to external services and to do this they use some form of a meshing network. These networks right now are done through container networks, such as the Docker Overlay Network, the Kubernetes "POD" system, linkerd, serf, and a multitude of other systems like istio. It is a space or area of concern, because there is no singular approach to all these differing container services right now. So all of them are vying to be the one that wins.
The issue that I find here is that you're looking in an English dictionary while these are technical networking / operating system terminologies and that dictionary will not help you in this namespace.
This page might help you a little more: https://en.wikipedia.org/wiki/Microservices
Also: Service being, a backend or software that performs some action based on an input / output
Okay, microservices look like what used to be called agents. For their communications there have been various efforts at ways to define data objects, complete with a registration hierarchy (that is, a case of public naming) and an inheritance hierarchy (roughly like some of inheritance in some cases of object oriented software). So, we had object request brokers, CORBA or some such. And we had the ISO/OSI CMIS/CMIP where ISO maybe abbreviates international standards organization, where OSI may abbreviate something in French, where some international telecommunications group, maybe part of the UN, was involved, and where CMIS abbreviated common management information system, and CMIP, common management information protocol, all mostly aimed at computer and network system monitoring and management. The work was somehow close to some old Unix work with management information base and ASN.1 -- abstract syntax notation version 1. Whew!
Okay, if there are to be lots of such microservices, as you nicely described, then they will want to be able to communicate. So, maybe they will want to use JSON (as I understand it, essentially just name-value pairs -- from Google, JavaScript Object Notation and, thus, maybe more than just name-value pairs), other mark up languages,
https://en.wikipedia.org/wiki/List_of_document_markup_langua...
etc. But we'd have to suspect that there needs to be some common global standards and data definitions.
Okay. Gee, in the past I went through a lot of that CMIS/CMIP, ASN.1, etc. stuff, wrote some internal papers, wrote some software, etc. so was totally torqued at microservices without definitions. Right, maybe the hidden secret is that we're supposed to retreat to Wikipedia for the undefined terms and acronyms -- bummer.
But, I still wonder: How popular, pervasive, important, practical so far are microservices? E.g., are they a lot like agents for system monitoring and management? Where are microservices getting to be important.
No, I'm not trolling. And with your description of microservices, we're making progress here.
Full Disclosure: work at IBM, but not in Watson.
