Query takes more time from user interface or in a network

I execute a pretty big sql query which joins multiple tables and I reviewed indexes on all these tables. I am happy with the result when I run the query using SSMS in the server locally i.e., where my SQL Server database is installed. It takes 4 seconds to get around 17000 records. If I run the same query in a network or from my desktop using SSMS i.e., i connect to the above mentioned SQL Server using SSMS, it takes more than 60 seconds. Not sure how to solve this. If someone could help me, it will be of great help.

Thanks.

Is the server local to you or installed in another office?

When you run the query on your machine enable the Client Statistics (second row of buttons at the top, next to the Actual Execution Plan button). What is the output from this?