Is it ok if I do this with my guinea pigs? or will it be stressful for them?

It's completely fine :) we have a guinea pig nursery and have babies al the time we pick them up like 10000times a day and they love it's moving them around causes no stress unless UR hurting them in some way