You've hit one of the classic problems artists face when working in 3d...
Detail. I used to work in SU a lot and encountered it often.
I would model every little detail and thus my models would have more geometry than they really needed.
Usually, a model is built for a scene, and a scene, unless animated, will have a view point.
The viewpoint will determine where the most detail needs to be based on the observer.
Details further in the back would be accomplished with materials or images applied to simpler geometry.
So all those stone supports or the windows further back would be done with flat imagery,
giving the look of detail but not the actual geometry, which is what is slowing down your process.
Even with all of that, you will still hit some limits if you create entire cities in SU.
SU is not really optimized for large sets of data/geometry.
Other 3d apps are better at handling that, like Blender, but not as easy or fun to use.
The alternative would be to model lots of smaller elements in SU and then assemble them in Blender and render them there too.
Depending on the version of SU that you are using, it may not even take advantage of 64 bit and thus all of the ram that you have.
That's just my bit of input based on doing some of this sort of thing in the past.
Hope that is helpful.
One thing I found helpful... 3d is a tool to assist, not an end in itself.
That said, I wasn't creating game assets or doing animation.
So my uses and needs may differ from yours.
I almost forgot to address this...
I think your specs are plenty good.
You are possibly just hitting the 32 bit limits of SU.
It's always been a problem when the geometry gets really dense/complex.
It may be 64 bit now, i'm not sure, as I use an old version of SU pro.