I "discovered" the valve WebJASPIOptionalAuthenticator in the JBoss AS (7.1.1, 7.1.3) source code. It doesn't seem to be documented anywhere, but judging from the code and from trying out it appears to do authentication for unprotected resources.

The comment above the class says that authentication of unprotected resources is optional, but I couldn't find a reference to this in the source code. Regardless, all other JASPIC/JASPI/JSR 196 implementations that I tested (GlassFish, Geronimo, WebSphere) always call the SAM for unprotected resources. See:

/**
 * <p>
 * This class implements a JASPI authenticator for unprotected resources. In the JASPI Servlet profile, authentication
 * for unprotected resources is optional but it is still allowed. When performed, the JASPI authentication modules must
 * grant access to the unprotected resources irrespective of the caller, which may be anonymous (i.e, no security info
 * supplied).
 * </p>
 *
 * @author <a href="mailto:sguilhen@redhat.com">Stefan Guilhen</a>
 */
@SuppressWarnings("unused")
public class WebJASPIOptionalAuthenticator extends ValveBase {

Additionally, while the class is called when configured in jboss-web.xml for unprotected resources, it doesn't seem to actually work. From the source code, it's clear that the callbackhandler isn't being processed:

  boolean isValid = sam.isValid(messageInfo, new Subject(), messageLayer, appContext, cbh);
  if (isValid) {
      WebLogger.WEB_SECURITY_LOGGER.debugf("JASPI validation for unprotected request context %s succeeded", request.getServletPath());
      sam.secureResponse(messageInfo, new Subject(),  messageLayer, appContext, cbh);
  }

Compare this to the corresponding fragment in WebJASPIAuthenticator, which does process the callbackhandler:

       if (sam != null) {
            result = sam.isValid(messageInfo, clientSubject, messageLayer, appContext, cbh);
        }

        // the authentication process has been a success. We need to register the principal, username, password and roles
        // with the container
        if (result) {
            PasswordValidationCallback pvc = cbh.getPasswordValidationCallback();
            CallerPrincipalCallback cpc = cbh.getCallerPrincipalCallback();

            // get the client principal from the callback.
            Principal clientPrincipal = cpc.getPrincipal();
            if (clientPrincipal == null) {
                clientPrincipal = new SimplePrincipal(cpc.getName());
            }

            // if the client principal is not a jboss generic principal, we need to build one before registering.
            if (!(clientPrincipal instanceof JBossGenericPrincipal))
                clientPrincipal = this.buildJBossPrincipal(clientSubject, clientPrincipal);

            this.register(request, response, clientPrincipal, authMethod, pvc.getUsername(),
                    new String(pvc.getPassword()));

            if (this.secureResponse)
                sam.secureResponse(messageInfo, new Subject(), messageLayer, appContext, cbh);
        }