Yes. Many fellowships support work and other experiences in the U.S. and abroad and are open to international students. Those with a U.S. location will require U.S. work eligibility.
The Center for Undergraduate Engagement provides basic visa advice for study abroad candidates. Whether you need a visa to visit a third country almost always depends on your nationality, not your status as a student in the U.S. You will have to look at the website of the embassy of the country you wish to visit to see their requirements for visitors. Columbia is conveniently located in New York City where you can easily visit the consulates from around the world if you do need to apply for a visitor visa.
Remember to have your I-20 on your person and not packed away in checked luggage since you go through inspection before claiming bags. That said, if you forget your I-20, you should ask the immigration official for an I-515 upon entry, which will allow your admission in F-1 status for 30 days. With an F-1 visa you are normally admitted to the U.S. For 'duration of status' (D/S,) but if you are missing the I-20 it is most likely you will be admitted for 30 days with an I-515.
If you are leaving the U.S. during your program of study, you need an unexpired passport, unexpired F-1 entry visa, and an I-20 which has a travel signature obtained from the International Students and Scholars Office (ISSO) on page 3 that is less than 12 months old at the time that you return. Please read Travel and Re-entry in F-1 Status for more information.
An entry visa in your passport is relevant ONLY to your eligibility to be admitted to the U.S., not for remaining in the U.S. What matters while you are in the United States is maintaining your F-1 status for the duration of your academic program as reflected on your I-20. If your entry visa expires while you are in the U.S. you will have to obtain a valid one before re-entering the U.S. from a trip abroad. You cannot renew a visa in the U.S. but will have to apply for a new one at a U.S. consulate abroad, most likely in your home country.