One thing you could do is involve HMACs. Basically, give the user a key that they use to sign the request (basically sign a hash containing the url, user id, requested options). You can then use the HMAC to verify that the user did request a video be transcoded, and website visitors cannot fiddle with the request parameters without invalidating the HMAC.

The problem is that this has to be done server side because the key needs to be secret (thus, this can't happen in javascript in the client), which breaks the whole idea you're selling (API-less transcoding).

You could also provide your clients with urls by having them submit the request on your site (obviously, behind a login). The user would then use that url in the src of their video tag. Again, this method goes against your selling point.