It often refers to programmers that develop things that other programmers use to construct applications. A systems programmer may work on compilers, operating systems, framework libraries, etc.

They will also often work with lower-level abstractions, such as threads, so that the higher-level developers need not think about them in their own designs.