Don't get me wrong though, it won't be a path of roses, aws error messages sometimes make no sense whatsoever and require some googling to figure out, typos are NOT your friend!
Luckily Google usually led me to someone trying to set something up similar and I could logic it out from there. My #1 thing that I wish someone had told me was that if you make a Ajax call to your web API and the browser spits out a CORS error, check, triple check the url and what kind of call you are making (POST, GET, etc) because at least on my setup mistakes in those actually shows up as a CORS error instead of your typical 404 error.
I also bit the bullet and figured out how to use one of the serverless sites to deploy the AWS API+lambda+dynamodb one one fell swoop using a .yml file. But first I did it all by hand via the AWS gui so I could get some idea of how it worked. That .yml file is a whole other rabbit hole as the error messages are pretty obtuse at times.
Edit: One thing to watch for on cost is the dynamodb, by default they reserve 25 total transactions per second on the free tier. If you try to increase the performance by reserving much a whole lot more, like 1000 tps per table/index, that will get pricey right quick. So either leave it at 25 and recognize that if you add an index things might get throttled while it builds the index, or do what I did, which is I went ahead and switched it to the pay as you go model since $0.20 per million transactions is plenty cheap and I'm not even flirting with that 20 cents on my proof of concepts that span 5 different web domains.