A river/stream/spring will start/occur any place where the water table is higher than the landscape. Water will continue to flow downhill as long as the water table is above the landscape, the water will flow downhill. There are lots of apparent exceptions, but those are usually related to changes in ground permeability (which affects the water table) more than any actual exceptions.

The most common places for the water table to be high are where there's lots of water, such as below snow caps on mountains. Mountains tend to accumulate more rainfall than plains, and rivers will often start there. However, if all of the water for a river were required to come from high sources, there would be very few rivers! What normally happens is that water falls all over land and soaks into the soil. That water forms the water table and tries to find a uniform level. When it impact the surface is when it becomes visible surface water. It's all of that water stored in the water table that makes rivers get larger as they flow along.

Apparently weird rivers happen a lot. A fairly common problem is disappearing rivers. Rivers flow along, collecting tributaries, and growing nicely. Then they hit a sand area, like an old dune field. The river shrinks dramatically, sometimes disappearing entirely. It may suddenly appear miles away from the sand. A river in karst terrain can just fall into a cave and be gone. It may appear later or it may not.