A median filter looks at all of the pixels in an area around a given pixel and then selects the one that's most likely to occur (the implementation is to collect all of the pixels in an area around the current one into a 1D array, sort those pixels, and then use the one in the middle of the array as the result). For shot noise (single pixels that vary wildy from the local average) or excess detail, a median filter will mostly remove the noise without too much loss of detail and usually without smoothing the edges as a simple blur would.
The notion of a shader is pretty simple: it takes input information such as color lists and height values, and uses those inputs to produce an output color. A shader might color the output based on the difference in angle between a light source and the local surface direction (the result looks like a gray lighted image), it might pick a color from a list based on altitude, it might search the local area to see how much sky is visible, or even combine those (e.g. the basic shader in Wilbur uses a pair of color lists combined with lighting information). A shader tree allows for combining the results of multiple shaders into a final results (e.g. a color list shader can be multiplied with a lighting shader to get something similar to the default Wilbur shader). A shader tree implementation typically allows the users to select the types of shaders and the operators used to combine them, including allowing different shaders for different inputs (e.g. one shader for land and one for sea). It's a flexible solution, but takes a bit of effort to implement.
As far as the description of the sort of coastline processing that you're interested in, I would recommend trying a vector-based solution. Trace along the coastline and then increase the fractalization for the vector paths. I know this is doable in something like CC3 if you use fractal polygons to trace your continents. CC3 is, by design, a fixed-scale implementation and has some limits on the total number of nodes in a polygon (the limit is conveniently similar to the actual Windows limitations). Other than that, though, it would be pretty straightforward, I think. Then you can output the coastline to an image if you'd like further processing. There is the manual tracing step that's in there, though. I think that Illustrator or Inkscape have similar functionality and may also be able to do the auto-trace operation.
I haven't used the gradient in The GIMP, sorry, so I can't really offer much in the way of assistance on that front, I'm afraid. The V2 shader in Wilbur allows a color list (a gradient with an awful user interface) to be used for the base equator-to-pole coloring. On my far-future list of toys to incorporate in Wilbur and FT is the notion of latitude values. The user would get a guide line to set at a particular latitude and then can attach a value to that latitude based on the current context. This feature would be useful, for example, to set a basic equator-to-pole gradient for rainfall or temperature. It would also be useful for something like the V2 shader, where the base color gradient could be directly specified on the map via guide lines.
I haven't gotten around to making a context-based help file for Wilbur, sorry. I usually hint that someone could do that by taking the (admittedly elderly) documentation on the Wilbur web site, chopping it into web pages, and then naming the pages according to the context-sensitive names shown in the error message. For example, hitting F1 on the main page brings up a notice that the help topic ID 131200 is missing and that it even tried to look for an HTML file named "[installdir]htmlhelp\W0020080.html" (other dialogs have different IDs, which allows for a hyperlinked system to be developed). I am trying to avoid mucking about with the documentation file because I'll update it and I have far too many things that I'm behind on to put any time into that activity. There are whole features in Wilbur that I haven't used in many years and there are features on dialogs that I also haven't used in a long time. I don't necessarily remember that they all do, either (I can look at the source and determine what, but why I would want to do that may still remain a mystery).
When I did the parts above, I started from the blue and green JPEG that you posted and then I converted it into a black and white mask GIF that I loaded into Wilbur as a selection and I drew a mountain mask to use as a second selection. (note that Wilbur will rescale your input selection images to map the whole world space, so it's easy to shrink things for rough processing and expand for later processing). I set the whole world to -50, then loaded the coast selection, set things to 100, then loaded the mountain selection and set things to 500. Now I had a basic two-plateau world on which to apply iterations of resize/noise/fill basins/incise flow/precipiton activities.