It doesn't appear like they worry a lot about the transition (other than give a warning on the back cover). Disc failure is probably the last of their concerns anyway, and to be honest, problems appear to be quite rare.

E.g., makers don't try to put the transition in between two scenes, or in the case of TV-series, in between two episodes,. They generally occur somewhere in the middle of a scene, causing a slight interruption in the video. So it seems they just let the break fall where it may.

If you use decent mastering software to create your own DVD's, then the layer break can be anywhere you want it to be. As to TV episodes, it could be almost impossible to get a clean layer break, as the disk is 'chocka' block with goodies, so hard to program in an unobtrusive layer break.
But I assume most people just use the lazy. fill up one layer then go on to the next approach.
I also find that you get less inturruption if they use the "fill to the outside then turnaround and work back into the middle" method as opossed to the "move the head back to the middle and start from there again" method.

As specise_8472 says, disc authors may choose where to put the break, obviously limited by the amount of data in a single stream they must put on the disc

eg if on your 8.5Gb disc you have a film that's 5Gb long there has to be a break *somewhere* in it since it won't fit on one layer. If the film is 8.5 Gb long you have no choice but the put it slap bang in the middle, if it's 7.5Gb you can put it anywhere between 3.25Gb in to the film and 4.25Gb in.

With TV they should (nearly) never have to put a pause in the middle of an episode since there is never a single episode so big it must stretch over more than half the disc and the break can be put between eps. The only exception is if the episodes together almost totally filled the disc and there was no way of grouping them so they all sat in one layer or the other eg, in crudest terms, 5 episodes at 1.7Gb each: 2 in one half fills up 3.4 of the 4.25 available and the fifth episode won't fit on either layer in full so would have to be split... sticking to an even number of episodes per disc would cure even this possibility 99.9% of the time.

The better authoring houses try to put a required break in a pause in the picture and music to minimise disruption but some just don't seem to care.

There is also the issue of the pause being much longer if the disc is authored inner edge->outer, inner edge->outer again rather than the more common inner edge->outer, outer edge->middle since the laser only has to re-focus not re-focus and more across the disc in the latter case.

I don't notice layer transitions that much, but when they author it right in the middle of a key scene it's really hard not to notice it. The layer transition on the "Pieces" DVD from Grindhouse Releasing for example is smack in the middle of a crucial scene, and also seems that it takes far longer to change the layers on that particular disc.

I've been out of the 'biz for a couple years, but as I recall... 
(re DVD spec, dunno about HD formats)

CSDL/PTP is quite rare on DVD Video - when it does tun up it's usually special situations such as a concert program where more multiple angles were desired than could be accommodated in a single program stream. Possibly also a convenient option for a studio that's directly converting two similar SL products into a single SS-DL unit (e.g. two separate P&S/WS releases, or a flipper, of the same program.)
RSDL/OTP has been the defacto standard since the DL process became reliable, tho PTP is much more common with DVDROM productions as random access is more efficient. 

The RSDL rules are that L0 data size must be larger than L1 (as you can't track back beyond the inside limit of L0 for reading L1), the break must occur only on a cell boundary, and that cell must be at the end of a disc ECC block.  It's a complex & unforgiving item, which fortunately good authoring software calculates and shows you the valid frames where the break marker can be set, making this nearly point&click. Hopefully one of the options lands on a nice transition point. If not, you have to shuffle program elements around so (hopefully) pushing something into a good spot. The more full to the disc capacity you are, the fewer the options.

A delay in layer change can be a combination of replication variations (transparency of L0 being at the edge of spec, eccentricity of L1, or/and several other physical factors), or issues with a specific player; dirty lens, weak head amp - anything that will inhibit attaining focus lock.  A small read-ahead buffer will exaggerate the layer change too (one of several reasons PS3's do changes so transparently.)
What people might be noticing in terms of very long delays is probably often just badly organized data, i.e. the author didn't examine the on-disc layout and relocate the most accessed elements within each VTS to be adjacent (and on a single layer), so long head seeks must be done to access the different program elements - and even menus.  (happens more often when they use the VMG to contain main elements of a complex layout, and not using VTS bridging from a common root selection area.)  Sometimes long seeks are unavoidable, but should be placed so they are on (hopefully) seldom used elements.

Authoring is really %70/30 art/science.