There are many different ways that HPC can be provided so it doesn't always hold true that what is good for one group is also good for the next. Some universities already provide shared HPC clusters that new faculty can use in which case it might not be necessary to build your own. Some groups also make use of supercomputer facilities like teragrid (replaced by XSEDE https://www.xsede.org/home ) so don't need a lot of power in their own labs. Others are starting to make use of Cloud service providers to do their high performance computing (eg. http://www.cyclecomputing.com ).
If you do decide to purchase your own cluster then there are different ways to go about doing it. There are vendors that sell entire turn-key clusters. That is, they are pre-built at some factory and shipped with software pre-installed. Others sell the hardware and then you can pay a separate fee to them (or to some other company) to come out and physically assemble the cluster and install software. Other groups, order only the hardware and then do the physical assembly and software installation themselves.
The other consideration, if you do purchase your own, is the physical environment required for a cluster. Any significantly sized cluster has to be in a special room that provides adequate cooling and power. Some universities have shared "co-location" datacenters (machine rooms) that faculty can use. In that case, sometimes they have strict policies in terms of what hardware you can bring in (some insist on particular vendors). Other times they can be very flexible.... it all depends on the policies at the particular university. If you have to build your own machine room, then the cost can be quite high to bring in adequate electrical power and air conditioning. This usually has to be done in collaboration with the Universities physical facilities department.