Ok funny guys, make way for real news.
PROGRESS!
Today I had to go on a 40-50 mile trip and something had been gnawing away in the back of my mind for weeks now, almost completely on a hunch with no real troubleshooting or theories behind it before leaving on the trip I took out the new coil pack that I fitted some time late last year and put the original 15 year old one back in. I had kept the original as at the time fitting the new coil pack made no change either for better or worse, so I thought it would be wasteful to throw the old one out if it was still good. (I later discovered that it was the rear bank plug leads that were faulty at the time, NOT the original coil pack)
Well blow me down if this thing isn't running almost normally now.

There's an enormous difference in performance from before, not quite perfect but on a scale of 1-100 I would say performance has gone from 40-50% of normal to about 85-90% of the best I've ever seen out of it. As soon as I started it it settled into a steady miss free idle with no visible rocking of the engine - in the last month or so there has been a miss every second or two at idle where you could clearly see the engine rock with each miss.
Pulling at lower RPM is hugely better and when you kick it down it really pushes you back in your seat now.
So what is going on guys ? Have I been the unfortunate victim of a new replacement coil pack failing on me in just over 6 months ? Or have I simply disturbed some faulty wiring ? All I did was pop the bonnet, remove the top plastic engine cover, unbolt the coil pack, put the new one in (same boots off the old coil pack used) put the cover back on and start the engine and the idle was perfect from the get go. I didn't disconnect the battery or mess with anything else, and I still have my dodgy negative terminal in place.
Performance did sag slightly after 50 miles of driving but only a little bit. I would put some of that down to the changes in torque converter lockup strategy with temperature - when the gearbox oil is fully warmed up the ECU more aggressively uses lock up (disabling half throttle TC unlock) and also doesn't kick down as easily which can make it feel like there isn't quite as much oomph, but if you manually change down the power is still there.
Another possible factor is the inlet air temperature gets quite high after the engine bay is really thoroughly heat soaked - even on a 17 degree day like today the inlet air through the butterfly is around 45 degrees by the end of a long trip which will reduce the volumetric efficiency a bit. (It could have done with an intercooler!

)
I'm not even convinced that the original coil pack is 100% but its running one hell of a lot better than the new one, unless it really is just a case of disturbed faulty wiring. I wiggled the coil pack primary plug and wires on the left hand end of the coil pack vigorously after the old coil pack was fitted and I couldn't get it to miss, so if its faulty wiring its not at that plug.
When I had the manifold out to do the fuel pressure regulator the other day the wiring loom did get tugged on and moved around quite a bit, so any dodgy wiring might have been provoked at the time...but in all the testing and work that I've done over the last 6 months I've never found any actual evidence of poor/intermittent wiring.
I even checked the strength of the spark from each coil only a month ago, and all were able to easily jump 20mm, so I found no evidence of a weak spark, but clearly there must have been a weak spark causing a misfire under load. Again.

(already fixed that problem once with new plug wires last year)
The lean oxygen sensor reading under wide throttle I reported recently makes sense now too - unburnt oxygen from the misfiring cylinder(s) passing through to the oxygen sensor. That and the intermittent miss at idle were the only clues that something was wrong with the ignition. Again.
I guess we now wait and see how long this lasts and whether it goes bad or not, (I'm not doing any cartwheels yet) but we have a very large very obvious jump in performance after touching only a very limited number of things. It really can only be the new coil pack failing, or a loose/intermittent connection somewhere in the wiring loom.
If it is the coil pack you can see why I've been banging my head against a wall trying to diagnose this car - why would I ever expect that the brand new coil pack would fail on me in a few months ??
