In The Works – Amazon Aurora Serverless (opens in new tab)

(aws.amazon.com)

87 pointspolmolea8y ago33 comments

33 comments

Last week I created a small framework called lambdaphp[1].

My aim was to host a Wordpress or Laravel site on aws lambda without paying any monthly hosting charges. I got everything running (sessions, fs, request, etc) except of course I still had to use RDS and I think this takes care of it too. So now I can expect to run a full site which is only billed by the amount of resources consumed.

Of course my project was just for my own amusement but I think this is the way how it's going to be done soon or where Aws is heading. Seems pretty nifty!

[1] https://github.com/san-kumar/lambdaphp

meritt8y ago

AWS will likely add PHP support, yes but they absolutely will not do what this project does: Running a NodeJS http server that launches a PHP binary and runs local scripts.

superasn8y ago

That's just one way to make it happen until we get proper support. I wrote to AWS support and they said that they will consider PHP support for AWS lamba as many people have requested it too (did not give me an ETA though).

The funny thing is the response time is quite good despite running it through a NodeJS server that launches a PHP binary (340ms, faster than 98% of the sites as per pingdom[1]).

[1] https://tools.pingdom.com/#!/ex9izm/https://www.lambdaphp.ho...

1 more reply

sologoub8y ago

Really wish "serverless" also meant that it can work with AWS Lambda efficiently. As is, each function would try to open a connection, making the overall overhead extremely high and stressing DBs.

k__8y ago

Can't you open the connection outside of the function?

So as long as the function is hot, it won't reconnect.

mjb8y ago

That's right. Make connections to databases (and most other things) the first time your Lambda handler runs, and stash them in a static/global variable for re-use on future runs. That allows you to amortize the cost of forming the connection over many executions of your function, which improves latency, reduces cost, and reduces load on the backend.

2 more replies

inopinatus8y ago

Are you sure? That is, do you have specific knowledge of the implementation? Because:

> The endpoint is a simple proxy that routes your queries to a rapidly scaled fleet of database resources.

That doesn’t seem to preclude a multiplexing proxy a la PgBouncer.

sologoub8y ago

I don't think it is, just wishing it was.

nleach8y ago

Is that true? The default limit on concurrent function executions is 1000. The existing Aurora (MySQL) should be able to handle cycling through those connections without issue.

sologoub8y ago

It’s not that the DB servers can’t handle it, it’s that establishing a connection is slower than re-using an existing one.

You also forgo certain optimizations within the DB designed to make fetching things for the given connection/scope faster, such as temp tables.

brootstrap8y ago

3 months later... AWS announces serverless, databaseless database system.

ignoramous8y ago

Which might be when their marketing team decides to re-launch S3?

noobiemcfoob8y ago

I'd merrily laugh at that ad :)

maxxxxx8y ago

And then "cloudless"?

dhd4158y ago

Given the existing architecture of Aurora, the independent scaling of CPU and storage seems pretty straightforward. What is much harder to scale up and (especially) down is a warmed-up buffer pool which is critical for consistent query performance. I wonder if that is what they mean in the article when they say that scaling happens on a "pool of 'warm' instances". If so, I'd be very interested in more details on how that works.

bpicolo8y ago

I wonder how the cost differs, given they have to keep hot standbys? Perhaps a lot of behind-the scenes prediction to make it cost effective? Takes a while to create a standby from scratch for large DBs.

Either way, sweet tech. Seems like a fun thing to build

zwily8y ago

Aurora doesn’t work that way - it has one storage layer shared by all the nodes. Adding a new replica is very fast no matter your data set size.

bpicolo8y ago

Ahh, makes sense. Thanks for the clarification there!

mrep8y ago

Is this for the master or the read replicas?

If it is for the master, that would be amazing and I would wonder if you can scale up past the 1 instance size max (currently r4.16xl)

drej8y ago

Cool... but.. how?

strong_silent_t8y ago

So, if I'm understanding right, it is using the database server instances as the current Aurora does. You just aren't responsible for managing them, the service makes those decisions within the spectrum of Aurora's capabilities and can change it in a very small time increment.

EDIT: pretty impressive though, looking at James Hamilton's criteria for "Automatic Management and Provisioning" here: https://www.usenix.org/legacy/event/lisa07/tech/full_papers/... , I think this addresses everything database related, as long as you are under the maximum capacity of Aurora.

odammit8y ago

Aurora already spreads each "10GB chunk of your database volume is replicated six ways, across three Availability Zones." It looks like from the diagram they are taking advantage of that fact and spinning down the instances that serve off those "disks"

jeffbarr8y ago

Read the post!

drej8y ago

I have, I'm just excited and baffled at the same time :-)

j / k navigate · click thread line to collapse

33 comments

superasn8y ago

Last week I created a small framework called lambdaphp[1].

Of course my project was just for my own amusement but I think this is the way how it's going to be done soon or where Aws is heading. Seems pretty nifty!

[1] https://github.com/san-kumar/lambdaphp

meritt8y ago

AWS will likely add PHP support, yes but they absolutely will not do what this project does: Running a NodeJS http server that launches a PHP binary and runs local scripts.

superasn8y ago

The funny thing is the response time is quite good despite running it through a NodeJS server that launches a PHP binary (340ms, faster than 98% of the sites as per pingdom[1]).

[1] https://tools.pingdom.com/#!/ex9izm/https://www.lambdaphp.ho...

1 more reply

sologoub8y ago

Really wish "serverless" also meant that it can work with AWS Lambda efficiently. As is, each function would try to open a connection, making the overall overhead extremely high and stressing DBs.

k__8y ago

Can't you open the connection outside of the function?

So as long as the function is hot, it won't reconnect.

mjb8y ago

2 more replies

inopinatus8y ago

Are you sure? That is, do you have specific knowledge of the implementation? Because:

> The endpoint is a simple proxy that routes your queries to a rapidly scaled fleet of database resources.

That doesn’t seem to preclude a multiplexing proxy a la PgBouncer.

sologoub8y ago

I don't think it is, just wishing it was.

nleach8y ago

Is that true? The default limit on concurrent function executions is 1000. The existing Aurora (MySQL) should be able to handle cycling through those connections without issue.

sologoub8y ago

It’s not that the DB servers can’t handle it, it’s that establishing a connection is slower than re-using an existing one.

You also forgo certain optimizations within the DB designed to make fetching things for the given connection/scope faster, such as temp tables.

brootstrap8y ago

3 months later... AWS announces serverless, databaseless database system.

ignoramous8y ago

Which might be when their marketing team decides to re-launch S3?

noobiemcfoob8y ago

I'd merrily laugh at that ad :)

maxxxxx8y ago

And then "cloudless"?

dhd4158y ago