UltraSPARC T1

Sun Microsystems' UltraSPARC T1 microprocessor, known until its 14 November 2005 announcement by its development codename "Niagara", is a multithreading, multicore CPU. Designed to lower the energy consumption of server computers, the CPU typically uses 72 W of power at 1.4 GHz.

Afara Websystems pioneered a radical thread-heavy SPARC design. The company was purchased by Sun, and the intellectual property became the foundation of the CoolThreads line of processors, starting with the T1. The T1 is a new-from-the-ground-up SPARC microprocessor implementation that conforms to the UltraSPARC Architecture 2005 specification and executes the full SPARC V9 instruction set. Sun has produced two previous multicore processors (UltraSPARC IV and IV+), but UltraSPARC T1 is its first microprocessor that is both multicore and multithreaded. The processor is available with four, six or eight CPU cores, each core able to handle four threads concurrently. Thus the processor is capable of processing up to 32 threads concurrently.

UltraSPARC T1 can be partitioned in a similar way to high-end Sun SMP systems. Thus, several cores can be partitioned for running a single or group of processes and/or threads, while the other cores deal with the rest of the processes on the system.

Cores

T1 microprocessor floorplan

The UltraSPARC T1's pipeline

The UltraSPARC T1 was designed from scratch as a multi-threaded, special-purpose processor, and thus introduces a whole new architecture for obtaining high performance. Rather than try to make each core as intelligent and optimized as they can, Sun's goal was to run as many concurrent threads as possible, and maximize utilization of each core's pipeline. The T1's cores are less complex than those of current high end processors in order to allow 8 cores to fit on the same die. The cores do not feature out-of-order execution, or a sizable amount of cache.

Single-thread processors depend heavily on large caches for their performance because cache misses result in a wait while the data is fetched from main memory. By making the cache larger the probability of a cache miss is reduced, but the impact of a miss is still the same.

The T1 cores largely side-step the issue of cache misses by multithreading. Each core is a barrel processor, meaning it switches between available threads each cycle. When a long-latency event occurs, such as cache miss, the thread is taken out of rotation while the data is fetched into cache in the background. Once the long-latency event completes, the thread is made available for execution again. Sharing of the pipeline by multiple threads may make each thread slower, but the overall throughput (and utilization) of each core is much higher. It also means that the impact of cache misses is greatly reduced, and the T1 can maintain high throughput with a smaller amount of cache. The cache no longer needs to be large enough to hold all or most of the "working set", just the recent cache misses of each thread.

Benchmarks demonstrate this approach has worked very well on commercial (integer), multithreaded workloads such as Java application servers, Enterprise Resource Planning (ERP) application servers, email (such as Lotus Domino) servers, and web servers. These benchmarks suggest each core in the UltraSPARC T1 is more powerful than the circa 2001, single-core, single-threaded UltraSPARC III, and at a chip to chip comparison, significantly outperforms other processors on multithreaded integer workloads.

Physical characteristics

The UltraSPARC T1 contained 279 million transistors and had an area of 378 mm². It was fabricated by Texas Instruments (TI) in their 90 nm complementary metal–oxide–semiconductor (CMOS) process with nine levels of copper interconnect.^[1] Each core has L1 16kB instruction cache and 8KB data cache. L2 cache is 3MB and there is no L3 cache.

Systems

Sun Fire T1000 server

The T1 processor can be found in the following products from Sun and Fujitsu Computer Systems:

• Sun/Fujitsu/Fujitsu Siemens SPARC Enterprise T1000 and T2000 servers
• Sun Fire T1000 and T2000 servers
• Sun Netra T2000 Server
• Sun Netra CP3060 Blade
• Sun Blade T6300 Server Module

Target market

The UltraSPARC T1 microprocessor is unique in its strength and weaknesses, and as such is targeted at specific markets. Rather than being used for high-end number-crunching and ultra-high performance applications, the chip is targeted at network-facing high-demand servers, such as high-traffic web servers, and mid-tier Java, ERP, and CRM application servers, which often utilize a large number of separate threads. One of the limitations of the T1 design is that a single floating point unit (FPU) is shared between all 8 cores, making the T1 unsuitable for applications performing a lot of floating point mathematics. However, since the processor's intended markets do not typically make much use of floating-point operations, Sun does not expect this to be a problem. Sun provides a tool for analysing an application's level of parallelism and use of floating point instructions to determine if it is suitable for use on a T1 or T2 platform.^[2]

In addition to web and application tier processing, the UltraSPARC T1 may be well suited for smaller database applications which have a large user count. One customer has published results showing that a MySQL application running on an UltraSPARC T1 server ran 13.5 times faster than on an AMD Opteron server.^[3]

Virtualization

T1 is the first SPARC processor that supports the Hyper-Privileged execution mode. The SPARC Hypervisor runs in this mode, and it can partition a T1 system into 32 Logical Domains, each of which can run an operating system instance.

Currently, Solaris and Linux are supported, and FreeBSD support is under development.^[4]

Software licensing issues

Traditionally, commercial software suites like Oracle database charge their customers based on the number of processors the software runs on. In early 2006, Oracle changed the licensing model by introducing the processor factor. With a processor factor of .25 for the T1, an 8-core T2000 requires only a 2-CPU license.^[5]

The "Oracle Processor Core Factor Table"^[6] has since been updated regularly as new CPUs came to market.

In Q3 2006, IBM introduced the concept of Value Unit (VU) pricing. Each core of the T1 is 30 PVUs (each T2 core is 50 PVUs, and T3 is 70 PVUs) instead of the default value of 100 PVUs per core.^[7]

Weaknesses

The T1 only offered a single Floating Point unit to be shared by the 8 cores, limiting usage in HPC environments. This weakness was mitigated with the follow-on UltraSPARC T2 processor, which included 8 floating point units, as well as other additional features.

The T1 was only available in uniprocessor systems, limiting vertical scalability in large enterprise environments. This weakness was mitigated with the follow-on "Victoria Falls",^[8] commercially known as the UltraSPARC T2 Plus, as well as the next generation SPARC T3 and SPARC T4. The UltraSPARC T2+, SPARC T3, and SPARC T4 all offer single, dual, and quad socket configurations.

The T1 had outstanding throughput with massive numbers of threads supported by the processor, but older applications burdened with single thread bottlenecks occasionally exhibited poor overall performance. Single threaded application weakness was mitigated with the follow-on SPARC T4 processor. The T4 core count was reduced to 8 (from 16 on the T3), the cores were made more complex, the clock rate was nearly doubled - all contributing to faster single thread performance (from between 300% to 500% increase over previous generations.^[9] Additional effort was made to add the "critical thread API", where the operating system would detect a bottleneck and would temporarily allocate the resources of an entire core, instead of a 1 of 8 threads, to the targeted application processes exhibiting single threaded CPU bound behavior.^[10] This allowed the T4 to uniquely mitigate single threaded bottlenecks, while not having to compromise in the overall architecture to achieve massive multi-threaded throughput.

Application tuning

Leveraging the massive amount of thread-level parallelism (TLP) available on the CoolThreads platform can require different application development techniques than for traditional server platforms. Using TLP in applications is key to getting good performance. Sun has published a number of Sun BluePrints to assist application programmers in developing and deploying software on T1 or T2-based CoolThreads servers. The main article, Tuning Applications on UltraSPARC T1 Chip Multithreading Systems,^[11] addresses issues for general application programmers. There is also a BluePrints article on using the Cryptographic Accelerator Units on the T1 and T2 processors.^[12]

Case studies

A wide range of applications were optimized on the CoolThreads platform, including Symantec Brightmail AntiSpam,^[13] Oracle's Siebel applications,^[14] and the Sun Java System Web Proxy Server.^[15] Sun also documented its experience in moving its own online store onto a T2000 server cluster,^[16] and have published two articles on web consolidation on CoolThreads using Solaris Containers.^[17][18]

Sun has an application performance tuning page for a range of open source applications, including MySQL, PHP, gzip, and ImageMagick.^[19] Proper optimization for CoolThreads systems can result in significant gains: when the Sun Studio compiler is used with the recommended optimization settings, MySQL performance improves by 268% compared to using just the -O3 flag.

Contemporary and subsequent designs

The "Coolthreads(TM)" architecture, beginning with the UltraSPARC T1 (with its positive and negative aspects), was certainly influential in the concurrent and future designs of SPARC processors.

"Rock"

The original UltraSPARC T1 was designed for single CPU systems only and is not capable of SMP. "Rock" was a more ambitious project, intended to support multiple-chip server architectures, targeting traditional data-facing workloads such as databases. It was seen as more a follow-on to Sun's SMP processors such as UltraSPARC IV, rather than a replacement for the UltraSPARC T1 or T2, but was canceled in the timeframe of Oracle's acquisition of Sun.

UltraSPARC T2

Formerly known by the codename Niagara 2, the follow-on to the UltraSPARC T1, the T2 provides eight cores. Unlike the T1, each core supports 8 threads per core, one FPU per core, one enhanced cryptographic unit per core, and CPU embedded 10 Gigabit Ethernet network controllers.

UltraSPARC T2 Plus

In February 2007, Sun announced at its annual analyst summit that its third-generation simultaneous multithreading design, code-named Victoria Falls, was taped out in October 2006. A two-socket server (2 RU) will have 128 threads, 16 cores, and a 65× performance improvement over UltraSPARC III.^[8]

At the Hot Chips 19 conference, Sun announced that Victoria Falls will be in two-way and four-way servers. Thus, a single 4-way SMP server will support 256 concurrent hardware threads.^[20]

In April 2008, Sun released 2-way UltraSPARC T2 Plus servers, the SPARC Enterprise T5140 and T5240.

In October 2008, Sun released 4-way UltraSPARC T2 Plus SPARC Enterprise T5440 server.^[21]

SPARC T3

In October 2006, Sun disclosed that Niagara 3 will be built with a 45 nm process. The Register, reported in June 2008 that the microprocessor will have 16 cores, incorrectly suggesting each core would have 16 threads. During the Hot Chips 21 conference Sun revealed the chip has a total of 16 cores and 128 threads.^[22][23] According to the ISSCC 2010 presentation:

"A 16-core SPARC SoC processor enables up to 512 threads in a 4-way glueless system to maximize throughput. The 6MB L2 cache of 461GB/s and the 308-pin SerDes I/O of 2.4Tb/s support the required bandwidth. Six clock and four voltage domains, as well as power management and circuit techniques, optimize performance, power, variability and yield trade-offs across the 377mm² die."^[24]

SPARC T4

The T4 CPU was released in late 2011. The new T4 CPU will drop from 16 cores (on the T3) back to 8 cores (as used on the T1, T2, and T2+). The new T4 core design (named "S3") feature improved per-thread performance, due to introduction of out-of-order execution, as well as having additional improved performance for single-threaded programs.^[25][26]

In 2010, Larry Ellison announced that Oracle will offer Oracle Linux on the UltraSPARC platform, and the port is scheduled to be available in the T4 and T5 timeframe.^[27]

John Fowler, Executive Vice President Systems Oracle, in Openworld 2014 said Linux will be able to run on Sparc at some point. ^{[28] [29] [30] [31]}

SPARC T5

The new T5 CPU features a total of 128 thread over 16 cores and is manufactured with a 28 nanometer technology.

Open design

On March 21, 2006, Sun made the UltraSPARC T1 processor design available under the GNU General Public License via the OpenSPARC project. The published information includes:

• Verilog source code of the UltraSPARC T1 design;
• Verification suite and simulation models;
• ISA specification (UltraSPARC Architecture 2005);
• The Solaris 10 OS simulation images.

References

External links

• OpenSPARC T1 and Specifications
• OpenSPARC Overview
• Sun's Big Splash by Linda Geppert, in IEEE Spectrum, January 2005
• Niagara, a 32-way Multithreaded SPARC Processor by Poonacha Kongetira, Kathirgamar Aingaran, Kunle Olukotun, in IEEE Micro, March–April 2005