If I do

<rich:collapsibleSubTableToggler collapsedIcon="chevronDown" expandedIcon="chevronUp"/>

the icons disappear.

The analogous setting works for collapsiblePanel.

RichFaces 4.0.0

Thanks