Thanks for the report, will investigate. There's been a change to SFTP in the latest restic release notes
Fix slow SFTP upload performance
https://github.com/restic/restic/releases/
I'll integrate that version in the next few days.
But if it's not fixed by that I'll have another look. 3 minutes sounds unusually long.