I recreate the view directory structure in the assets folder, but only for views where I need a complex (angular) UI. So I might have a file in assets/javascripts/users/show.js because the users#show view is more complicated than I want to do in rails.
If that asset file exists, I have a helper automatically load it into the head section of the view layout when the users#show action is hit. The helper itself is only a few lines of some obscure rails code that looks up the name of the controller, action, etc and looks up the files in the assets directory. If you match the view directory structure to the assets directory structure, it stays very organized and is all done by convention.
In the show.js file, I will create an entire mini angular app (one file). The show.html.erb view will just contain angular markup (and maybe some erb). It's that simple. If Angular needs some instance variables from rails, I just pass them in to js with a couple lines in a <script> tag at the beginning of the view.
If I need a custom directive, I put the templates for those in public/templates (there may be a better place for these, depending on what you're doing - for my use case, this was fine).
There are some downsides - you have to separately compile each js file. I suppose you could do them all together, but I like keeping them separate. But largely, you can lay angular on top of rails in a very unintrusive manner this way. It probably doesn't work for every situation - you have to keep your angular apps small or it gets complicated again. But it really lets you enhance the UI of a rails app without a lot of technical debt. Code reuse can be a challenge (organizing bits that are shared is kinda wonky), but often the angular apps are small and simple enough that this isn't really a problem. Repeat yourself a little. It won't kill you.
An example of where I've used it is to load a chart and some daily/weekly/monthly selectable data that is sortable by columns, by date, with pagination, search fields, etc. I make the data available via rails as JSON, and just have the angular app pull it all in for the user to manipulate.
Another would be a form that creates a query interface similar to the one used by google analytics to create custom segments.
There may be better ways to do all this, but this approach has served me well. It's a little rough, and I'm sure it could be improved. As always, YMMV.