Proxy call issue from the browser

ashvini · March 19, 2019, 11:11am

I have a unique and strange issue with Ory Oathkeeper proxy.

Proxy calls are working fine with a correct token but when a token is expired or wrong then I am not getting any response in the browser call.

But when I hit the same route with cURL then it gives the response:

{"error":{"code":403,"status":"Forbidden","reason":"Access token introspection says token is not active","message":"Access credentials are not sufficient to access this resource"}}

and bothe time in the oathkeeper logs it says correct:

INFO[1153] completed handling request                    measure#oathkeeper-proxy.latency=556985 method=GET remote="[::1]:60834" request=/users-api/user status=403 text_status=Forbidden took="556.985µs"
INFO[3718] started handling request                      method=OPTIONS remote="[::1]:56592" request=/users-api/user
WARN[3718] Access request granted                        access_url="http://localhost:3000/user" granted=true
INFO[3718] completed handling request                    measure#oathkeeper-proxy.latency=419740 method=OPTIONS remote="[::1]:56592" request=/users-api/user status=200 text_status=OK took="419.74µs"
INFO[3718] started handling request                      method=GET remote="[::1]:56592" request=/users-api/user
WARN[3718] The authentication handler encountered an error  access_url="http://localhost:4455/users-api/user" authentication_handler=oauth2_introspection error="Access credentials are not sufficient to access this resource" granted=false reason_id=authentication_handler_error
WARN[3718] Access request denied                         access_url="http://localhost:4455/users-api/user" error="Access credentials are not sufficient to access this resource" granted=false
ERRO[3718] An error occurred while handling a request    code=403 details="map[]" error="Access credentials are not sufficient to access this resource" reason="Access token introspection says token is not active" request-id= status=Forbidden trace="Stack trace: \ngithub.com/ory/oathkeeper/proxy.(*AuthenticatorOAuth2Introspection).Authenticate\n\t/home/ridham/go/src/github.com/ory/oathkeeper/proxy/authenticator_oauth2_introspection.go:119\ngithub.com/ory/oathkeeper/proxy.(*RequestHandler).HandleRequest\n\t/home/ridham/go/src/github.com/ory/oathkeeper/proxy/request_handler.go:100\ngithub.com/ory/oathkeeper/proxy.(*Proxy).Director\n\t/home/ridham/go/src/github.com/ory/oathkeeper/proxy/proxy.go:121\ngithub.com/ory/oathkeeper/proxy.(*Proxy).Director-fm\n\t/home/ridham/go/src/github.com/ory/oathkeeper/cmd/serve_proxy.go:219\nnet/http/httputil.(*ReverseProxy).ServeHTTP\n\t/usr/local/go/src/net/http/httputil/reverseproxy.go:197\ngithub.com/urfave/negroni.Wrap.func1\n\t/home/ridham/go/pkg/mod/github.com/urfave/[email protected]/negroni.go:46\ngithub.com/urfave/negroni.HandlerFunc.ServeHTTP\n\t/home/ridham/go/pkg/mod/github.com/urfave/[email protected]/negroni.go:29\ngithub.com/urfave/negroni.middleware.ServeHTTP\n\t/home/ridham/go/pkg/mod/github.com/urfave/[email protected]/negroni.go:38\ngithub.com/urfave/negroni.middleware.ServeHTTP-fm\n\t/home/ridham/go/pkg/mod/github.com/urfave/[email protected]/negroni.go:38\ngithub.com/ory/x/metricsx.(*MetricsManager).ServeHTTP\n\t/home/ridham/go/pkg/mod/github.com/ory/[email protected]/metricsx/middleware.go:207\ngithub.com/urfave/negroni.middleware.ServeHTTP\n\t/home/ridham/go/pkg/mod/github.com/urfave/[email protected]/negroni.go:38\ngithub.com/urfave/negroni.middleware.ServeHTTP-fm\n\t/home/ridham/go/pkg/mod/github.com/urfave/[email protected]/negroni.go:38\ngithub.com/meatballhat/negroni-logrus.(*Middleware).ServeHTTP\n\t/home/ridham/go/pkg/mod/github.com/meatballhat/[email protected]/middleware.go:136\ngithub.com/urfave/negroni.middleware.ServeHTTP\n\t/home/ridham/go/pkg/mod/github.com/urfave/[email protected]/negroni.go:38\ngithub.com/urfave/negroni.(*Negroni).ServeHTTP\n\t/home/ridham/go/pkg/mod/github.com/urfave/[email protected]/negroni.go:96\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2741\nnet/http.(*conn).serve\n\t/usr/local/go/src/net/http/server.go:1847\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1333" writer=JSON
INFO[3718] completed handling request                    measure#oathkeeper-proxy.latency=945779 method=GET remote="[::1]:56592" request=/users-api/user status=403 text_status=Forbidden took="945.779µs"
INFO[4016] started handling request                      method=GET remote="127.0.0.1:37218" request=/users-api/user
WARN[4016] The authentication handler encountered an error  access_url="http://localhost:4455/users-api/user" authentication_handler=oauth2_introspection error="Access credentials are not sufficient to access this resource" granted=false reason_id=authentication_handler_error
WARN[4016] Access request denied                         access_url="http://localhost:4455/users-api/user" error="Access credentials are not sufficient to access this resource" granted=false
ERRO[4016] An error occurred while handling a request    code=403 details="map[]" error="Access credentials are not sufficient to access this resource" reason="Access token introspection says token is not active" request-id= status=Forbidden trace="Stack trace: \ngithub.com/ory/oathkeeper/proxy.(*AuthenticatorOAuth2Introspection).Authenticate\n\t/home/ridham/go/src/github.com/ory/oathkeeper/proxy/authenticator_oauth2_introspection.go:119\ngithub.com/ory/oathkeeper/proxy.(*RequestHandler).HandleRequest\n\t/home/ridham/go/src/github.com/ory/oathkeeper/proxy/request_handler.go:100\ngithub.com/ory/oathkeeper/proxy.(*Proxy).Director\n\t/home/ridham/go/src/github.com/ory/oathkeeper/proxy/proxy.go:121\ngithub.com/ory/oathkeeper/proxy.(*Proxy).Director-fm\n\t/home/ridham/go/src/github.com/ory/oathkeeper/cmd/serve_proxy.go:219\nnet/http/httputil.(*ReverseProxy).ServeHTTP\n\t/usr/local/go/src/net/http/httputil/reverseproxy.go:197\ngithub.com/urfave/negroni.Wrap.func1\n\t/home/ridham/go/pkg/mod/github.com/urfave/[email protected]/negroni.go:46\ngithub.com/urfave/negroni.HandlerFunc.ServeHTTP\n\t/home/ridham/go/pkg/mod/github.com/urfave/[email protected]/negroni.go:29\ngithub.com/urfave/negroni.middleware.ServeHTTP\n\t/home/ridham/go/pkg/mod/github.com/urfave/[email protected]/negroni.go:38\ngithub.com/urfave/negroni.middleware.ServeHTTP-fm\n\t/home/ridham/go/pkg/mod/github.com/urfave/[email protected]/negroni.go:38\ngithub.com/ory/x/metricsx.(*MetricsManager).ServeHTTP\n\t/home/ridham/go/pkg/mod/github.com/ory/[email protected]/metricsx/middleware.go:207\ngithub.com/urfave/negroni.middleware.ServeHTTP\n\t/home/ridham/go/pkg/mod/github.com/urfave/[email protected]/negroni.go:38\ngithub.com/urfave/negroni.middleware.ServeHTTP-fm\n\t/home/ridham/go/pkg/mod/github.com/urfave/[email protected]/negroni.go:38\ngithub.com/meatballhat/negroni-logrus.(*Middleware).ServeHTTP\n\t/home/ridham/go/pkg/mod/github.com/meatballhat/[email protected]/middleware.go:136\ngithub.com/urfave/negroni.middleware.ServeHTTP\n\t/home/ridham/go/pkg/mod/github.com/urfave/[email protected]/negroni.go:38\ngithub.com/urfave/negroni.(*Negroni).ServeHTTP\n\t/home/ridham/go/pkg/mod/github.com/urfave/[email protected]/negroni.go:96\nnet/http.serverHandler.ServeHTTP\n\t/usr/local/go/src/net/http/server.go:2741\nnet/http.(*conn).serve\n\t33/usr/local/go/src/net/http/server.go:1847\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1333" writer=JSON
INFO[4016] completed handling request                    measure#oathkeeper-proxy.latency=2678217 method=GET remote="127.0.0.1:37218" request=/users-api/user status=403 text_status=Forbidden took=2.678217ms

What’s the solution over here?

hackerman · March 21, 2019, 2:29pm

You do get a response - it’s a 403 status code! There are currently no other mitigation strategies for unauthorized calls (e.g. redirecting the browser somewhere). We will add that though soon.

ashvini · March 25, 2019, 4:41am

@hackerman
Sorry, I was able to add only one screenshot. I am getting 403 but with an empty response.

whereas if I call it from Postman then it sends a response that Access credentials are not sufficient to access this resource.

In the console, I get CORS error but my response to Option request clearly allows the request.

hackerman · March 25, 2019, 10:23am

That’s what I said - you’ll get a 403 with an empty response body!

ashvini · March 25, 2019, 10:49am

so if I hit this url with postman I get JSON response and from browser I will get empty response.
I don’t think it should be expected behaviour as I have all my servers behind the proxy and if call fails from proxy it gives CORS errors+no response.

ashvini · March 25, 2019, 12:04pm

One more thing @hackerman,
If I give correct credentials in the proxy call, it executes successfully but if I give incorrect token it throws CORS error which is very unlikely as if success response don’t throw CORS then failure case also should not throw it

hackerman · March 25, 2019, 12:28pm

Did you enable CORS on the proxy side or at your upstream server?

ashvini · March 25, 2019, 1:21pm

I have enabled CORS on server and proxy is running on default settings.

Main configs are as follow

AUTHENTICATOR_OAUTH2_INTROSPECTION_URL=http://localhost:4445/oauth2/introspect \                                                
OATHKEEPER_API_URL=http://localhost:4456/ \
AUTHORIZER_KETO_WARDEN_KETO_URL=http://localhost:4466 \
DATABASE_URL=memory \
CREDENTIALS_ISSUER_ID_TOKEN_HS256_SECRET=changemechangemechangemechangemedo \
PORT=4455 \
./oathkeeper serve proxy

ashvini · March 25, 2019, 1:23pm

I don’t see any CORS options in the docs

hackerman · March 25, 2019, 3:56pm

Ok, in that case it is not possible for Oathkeeper to return the CORS headers. Since the request is denied, Oathkeeper does not call the upstream server (your API) which would set the CORS headers. You must enable CORS at Oathkeeper itself. This is documented in the help section of the Oathkeeper command:

$ oathkeeper help serve proxy
This starts a HTTP/2 reverse proxy capable of deciding whether to forward API requests or to block them based on a set of rules.

[....]

CORS CONTROLS
==============
- CORS_ENABLED: Switch CORS support on (true) or off (false). Default is off (false).
	Example: CORS_ENABLED=true

- CORS_ALLOWED_ORIGINS: A list of origins (comma separated values) a cross-domain request can be executed from.
	If the special * value is present in the list, all origins will be allowed. An origin may contain a wildcard (*)
	to replace 0 or more characters (i.e.: http://*.domain.com). Usage of wildcards implies a small performance penality.
	Only one wildcard can be used per origin. The default value is *.
	--------------------------------------------------------------
	Example: CORS_ALLOWED_ORIGINS=http://*.domain.com,http://*.domain2.com
	--------------------------------------------------------------

- CORS_ALLOWED_METHODS: A list of methods  (comma separated values) the client is allowed to use with cross-domain
	requests. Default value is simple methods (GET and POST).
	--------------------------------------------------------------
	Example: CORS_ALLOWED_METHODS=POST,GET,PUT
	--------------------------------------------------------------

- CORS_ALLOWED_CREDENTIALS: Indicates whether the request can include user credentials like cookies, HTTP authentication
	or client side SSL certificates.
	--------------------------------------------------------------
	Default: CORS_ALLOWED_CREDENTIALS=false
	Example: CORS_ALLOWED_CREDENTIALS=true
	--------------------------------------------------------------

- CORS_DEBUG: Debugging flag adds additional output to debug server side CORS issues.
	--------------------------------------------------------------
	Default: CORS_DEBUG=false
	Example: CORS_DEBUG=true
	--------------------------------------------------------------

- CORS_MAX_AGE: Indicates how long (in seconds) the results of a preflight request can be cached. The default is 0
	which stands for no max age.
	--------------------------------------------------------------
	Default: CORS_MAX_AGE=0
	Example: CORS_MAX_AGE=10
	--------------------------------------------------------------

- CORS_ALLOWED_HEADERS: A list of non simple headers (comma separated values) the client is allowed to use with
	cross-domain requests.

- CORS_EXPOSED_HEADERS: Indicates which headers (comma separated values) are safe to expose to the API of a
	CORS API specification.

Usage:
  oathkeeper serve proxy [flags]

Flags:
  -h, --help   help for proxy

Global Flags:
      --disable-telemetry   Disable anonymized telemetry reports - for more information please visit https://www.ory.sh/docs/ecosystem/sqa

ashvini · March 28, 2019, 7:31am

Thanks, however to make this work I had to disabled cors on my own server else it is not working.

Other thing is all the pre-flight requests are handled by proxy and not forwarded to the upstream which might be an issue for few use cases.

And for other requests if I keep cors enabled on upstream go server all proxy rewrites headers which browser rejects due to invalid values like “,”