The city's name as it is locally pronounced is "Asfi", which was Latinized as "Safi" and "Safim" under Portuguese rule. "Asfi" means flood or river estuary in Berber and comes from the Berber root "sfi/sfey" which means to flood, to spill or to pour. 11th-century geographer Al-Idrisi gave an apparently false explanation to the origin the name "Asfi" as he linked it to the Arabic word "Asaf" (regret); Asafi (my regret). He based this claim on a strange story about some sailors from al-Andalus who sailed to discover the other end of the Atlantic ocean but got lost and landed on some island where the natives captured them and sent them back on their ships blindfolded. The ships eventually ended on the shores of "Asfi" and locals helped the lost sailors and told them that they were two months away from their native land al-Andalus. Upon hearing this one of the sailors responded by saying: "Wa asafi" (Oh my regret). Al-Idrisi wrote that from that time the city carried the name "Asafi". This story is thought to be a legend and unlikely explanation of the origin of the name. Wikipedia
This warm abode of Safi is a delightful place to visit in Morocco. This city is located in the close proximity of a major city Marrakesh, Morocco. Marrakesh is bestowed with interesting set of attractions like Jardin Majorelle, Djemaa El Fna, Ali Ben Youssef Medersa. Also, to reach Safi, you'll have to take a flight to Marrakesh; so its not really a detour. And even if it is, the city is worth paying at least a quick visit. 1-2 days is an ideal duration of visit to Safi.
Yes,it is that simple really! Enjoy your trip! Keep travelling!