With Expression Blend it's quite easy to style your own custom ScrollBar to get the look you want.

In this example I wanted the corners to be rounded with some gradient. Looks great. Now let's see the horizontal version.

Looks just like the vertical version, just as we expected. Just to be sure everything is OK, let's see what it looks like with both.

This is definitely not what we expected!

When this first happened to me, I figured that I missed setting the transparency on some element within the ScrollBar style. After pulling much hair and performing some creative Internet searches (finding nothing on the subject), I came to the realization that I did not miss anything in the ScrollBar style. If this was true, where was this white box coming from? Although I did not explicitly declare a ScrollViewer, I suspected that one was getting created to house the scrollbars. I created a global ScrollViewer style and set the Grid background to Transparent.

<Style TargetType="{x:Type ScrollViewer}">
    <Setter Property="Template">
        <Setter.Value>
            <ControlTemplate TargetType="{x:Type ScrollViewer}">
                <Grid x:Name="Grid" Background="Transparent">
                    <Grid.ColumnDefinitions>
                        <ColumnDefinition Width="*"/>
                        <ColumnDefinition Width="Auto"/>
                    </Grid.ColumnDefinitions>
                    <Grid.RowDefinitions>
                        <RowDefinition Height="*"/>
                        <RowDefinition Height="Auto"/>
                    </Grid.RowDefinitions>
                    .................
            </ControlTemplate>
        </Setter.Value>
    </Setter>
</Style> 

...And the white box disappeared.

Just as expected!