The physics of pushing a longer, narrower shot column and the increasing pressure developed due to friction and constriction.
A 12 gauge has a bore just under .75 inches, a 20g just under .62 inches, and a .410, .41 inches (33% tighter than 20g, almost 44% tighter than a 12g).
To keep it to your .410 and 20g options, let's use Noah's recommendation of 1/2oz #9s. Just looked that up and #9s are .08" and you'd get roughly 180 pellets in 1/2 oz.
As is obvious, that 1/2oz would require a longer shot column to fit in the .410 shell than in the .615 (20g) shell. That puts more pellets in contact with the barrels... more friction. More friction means more pressure behind the shot column, yielding more recoil.
I'm not suggesting that this is the biggest contributor, I suspect it is relatively small... but, I also suspect between this and the little extra weight of the 20g vs the .410, you'd feel notably less recoil shooting a 20g with a .410 load (even stepping up to 5/8 or 9/16 oz).
That half ounce charge of TSS won't work in a 20 gauge, trust me we have tried. With TSS being extremely dense the shot columns are ultra short. 1/2oz of TSS in a 20ga wad takes up little to no room. 5/8oz in a 20ga will work but it requires some serious fillers and wad trimming to take up enough space to get it to fit. Most guys shooting payloads that light are stacking a 28ga wad inside of their 20ga wad to take up space.
For reference, to get 5/8oz of TSS to fit in a 3" 410 wad we have to use 3/4" of felt fillers. The 3" 410 wad is only about 65% full with 5/8oz of shot.
3/4 or 7/8oz in a 20ga is much easier to achieve.
The phenomena Henry describes may contribute slightly to recoil, but the major players in the equation are gun weight, velocity of the shell, and the overall payload weight including the wad and powder.
Felt recoil under 10 foot pounds is very little. You can get it down even further by slowing down your loads.
TSS #9 shot at 1300 fps gets 1.50" of gel penetration at 68.5 yards. It doesn't have to be going very fast to be effective. Most turkey loads are 1100 fps or less.
Say we slowed our 1/2oz payload down to 1200 fps, now we are at 6.7 foot pounds of recoil
Say we slowed our payload down to 1200 FPS and got the gun weight up to 6.5 pounds, now we are at 5.9 foot pounds of recoil.
If it were me, I wouldn't worry with gun modifications, and would just shoot the tristar the way it comes with the factory cylinder choke and 1/2 oz of TSS 9's at 1200 FPS. Hodgdon Lil Gun powder will work well for this.