For most people, falling off the wagon is a far bigger problem than optimizing their workout protocols. Therefore, the best time to do your workout is whenever you can fit it into your schedule best as part of a regular schedule -- such that you will not fall off the wagon.

If you are one of those lucky people who does not have a problem with this, then yes, early afternoon may be better than early morning. And early evening may be better than early morning.

Personally, I find it hard to warm up effectively for early morning workouts. I feel stiff, range of motion is not good, and mobility sucks.

Eric Cressey thinks a 4PM time is optimal for him, based on his experience and some data:

Time of day is also significant with respect to injury risk potential, as there's a diurnal variation in spine length due to the in- and outflow of the fluid within the intervertebral discs. At night, while you’re lying horizontally and the spine isn’t loaded, fluid enters the disc; when you wake up and start moving around, fluid begins to leave the disc. By the end of the day, you’re actually shorter!

Oddly enough, you’re safer in the "short" scenario. The increased fluid content present when you first wake up is associated with increased stiffness of the spine when you bend (because the discs have expanded). Because the muscles can’t just adapt to compensate for this alteration to spine length, the discs and ligaments take on more of the stress.

Body temperature is also lower upon rising, so range of motion (ROM) is compromised even further. As the day goes on and you move around more, body temperature increases and the fluid flows out of the disc, improving ROM and reducing ligament stress. McGill (2004) noted that in the morning, disc-bending and ligament stresses during forward flexion were 300% and 80% greater than when performed later in the day. Moreover, lumbar flexion ROM increases by 5-6° during this same time period (2).

Basically, you need to give your spine at least thirty minutes in the morning to "wake up," and I’d prefer that you wait at least three hours to train. This recommendation is largely based on the increase in body temperature that you experience in this time. The data from Armstrong presented above actually shows that maximal grip strength and body temperature peak at approximately the same time of day (~4PM). Speaking anecdotally, this is the time of day at which I seem to have my best training sessions. This would obviously be influenced by how early in the day you arise, though. If you’re up earlier, this time would likely be shifted forward a bit.

Due to schedule concerns, I usually start warming up at 6:30 pm and start my workout around 7. On occasion, if I hit a conditioning workout hard, then I will have trouble falling asleep. So, be careful of this.

Do whatever works best for you.

Men who workout in the morning before breakfast get a more potent adaptive response to their exercise due to increased physiological exertion from the fasted state.

Also, runners have been evaluated for effectiveness during various times of a the day, and sprinters had faster times midday, as opposed to morning or evening.

i heard James Fitzgerald (OPT) speak recently and he gave a pretty compelling argument for early-morning workouts. He acknowledged the "spinal compression" argument above, but felt it was outweighed by the tremendous impact of heavy training on circadian rhythms and specifically cortisol production.

I would say that this depends on your goals. If you are just trying to get some exercise in then whenever is convenient for you is best. Optimization is probably part of trying to attain elite levels of fitness.

If you are concentrating on leaning out (and still are able to get enough sleep!) then perhaps the morning is a good idea since it is in the fasted state and you get those benefits. I used to be a rower in college and while the morning training left me in awesome shape I had a hard time staying awake in my early classes and it messed up my sleep schedule.

Personally if I exercise after 5ishpm I can't go to sleep. I think my body has a heightened response to it. I tend to crossfit between 4 and 6 pm.

Whatever makes you most likely to stick to it. There are probably some fine arguments around being fasted, fatigue levels, hormonal response, etc, but a consistent routine that's 90% perfect trumps an inconsistent perfect routine in the long run.

I work out late at night simply because my job means I have to get up at 5am and I don't fancy working out before that!