Hi,
I’ve started to look into using a combo of Ory Kratos and Ory Hydra, and put an API Gateway in front of them, so my first step is to try to get to know Ory Kratos. For that reason I’ve installed Kratos and the Self-Service UI node in my Kubernetes cluster. Having spent some time trying to figure out how to configure Kratos and the Self-service node I think I’ve got the configuration kind of sorted out, but when I try to initiate a browser base login flow, I run into trouble. So far I’ve running everything through port-forwarded traffic as I don’t want to expose anything over the Internet yet. That will explain the use of the IP address 127.0.0.1 in the configurations and the description of the issue.
So, the flow is initiated:
Request URL:http://127.0.0.1:4434/self-service/browser/flows/login
Request Method:GET
Remote Address:127.0.0.1:4434
Status Code: 302
Request Headers:
Host: 127.0.0.1:4434
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:74.0) Gecko/20100101 Firefox/74.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Connection: keep-alive
Cookie: csrf_token=3ZL9ZXyHlxKkO0H6hx3vdzO/PqVA8Hk/FfL1jfmBtzk=
Upgrade-Insecure-Requests: 1
Response Headers:
HTTP/1.1 302 Found
Content-Type: text/html; charset=utf-8
Location: http://127.0.0.1:4435/auth/login?request=0fb8e59d-1d7d-4f54-b509-101540ac7384
Vary: Cookie
Date: Wed, 25 Mar 2020 19:45:04 GMT
Content-Length: 100
So, for the first request, everything seems to be OK, except maybe for a missing Set-Cookie header in the response?
Now, Kratos redirects to the UI:
Request URL:http://127.0.0.1:4435/auth/login?request=0fb8e59d-1d7d-4f54-b509-101540ac7384
Request Method:GET
Remote Address:127.0.0.1:4435
Status Code: 500
Request Headers:
Host: 127.0.0.1:4435
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:74.0) Gecko/20100101 Firefox/74.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
DNT: 1
Connection: keep-alive
Upgrade-Insecure-Requests: 1
Response Headers:
HTTP/1.1 500 Internal Server Error
X-Powered-By: Express
Content-Type: text/html; charset=utf-8
Content-Length: 10761
ETag: W/"2a09-zz7Y0rMegEgU3Fta4fm+mqMd+WQ"
Date: Wed, 25 Mar 2020 19:45:04 GMT
Connection: keep-alive
And the response is:
An error occurred
{
"response": {
"statusCode": 403,
"body": {
"error": {
"code": 403,
"status": "Forbidden",
"reason": "A request failed due to a missing or invalid csrf_token value.",
"debug": "The requested action was forbidden",
"message": "The requested action was forbidden"
}
},
"headers": {
"content-type": "application/json",
"set-cookie": [
"csrf_token=pNoPNQF60VWUyvylguSn7bdxG3055r3oC28LZEDfS2A=; Domain=127.0.0.1; Max-Age=31536000; HttpOnly"
],
"vary": "Cookie",
"date": "Wed, 25 Mar 2020 19:45:04 GMT",
"content-length": "210",
"connection": "close"
},
"request": {
"uri": {
"protocol": "http:",
"slashes": true,
"auth": null,
"host": "10.0.27.88:4434",
"port": "4434",
"hostname": "10.0.27.88",
"hash": null,
"search": "?request=0fb8e59d-1d7d-4f54-b509-101540ac7384",
"query": "request=0fb8e59d-1d7d-4f54-b509-101540ac7384",
"pathname": "/self-service/browser/flows/requests/login",
"path": "/self-service/browser/flows/requests/login?request=0fb8e59d-1d7d-4f54-b509-101540ac7384",
"href": "http://10.0.27.88:4434/self-service/browser/flows/requests/login?request=0fb8e59d-1d7d-4f54-b509-101540ac7384"
},
"method": "GET",
"headers": {
"Accept": "application/json"
}
}
},
"body": {},
"statusCode": 403,
"name": "HttpError"
}
So, apparently the Self-Service UI expects a csrf_token, which is not present in the request?
The logs for the UI node:
kubectl logs ory-kratos-selfservice-ui-node-57b9dc8789-n9zf6 -n ory
> [email protected] serve /usr/src/app
> node lib/index.js
Listening on http://0.0.0.0:4435
HttpError: HTTP request failed
at Request._callback (/usr/src/app/node_modules/@oryd/kratos-client/dist/api/adminApi.js:299:40)
at Request.self.callback (/usr/src/app/node_modules/request/request.js:185:22)
at Request.emit (events.js:321:20)
at Request.<anonymous> (/usr/src/app/node_modules/request/request.js:1161:10)
at Request.emit (events.js:321:20)
at IncomingMessage.<anonymous> (/usr/src/app/node_modules/request/request.js:1083:12)
at Object.onceWrapper (events.js:427:28)
at IncomingMessage.emit (events.js:333:22)
at endReadableNT (_stream_readable.js:1220:12)
at processTicksAndRejections (internal/process/task_queues.js:84:21) {
response: <ref *1> IncomingMessage {
_readableState: ReadableState {
objectMode: false,
highWaterMark: 16384,
buffer: BufferList { head: null, tail: null, length: 0 },
length: 0,
pipes: [],
flowing: true,
ended: true,
endEmitted: true,
reading: false,
sync: true,
needReadable: false,
emittedReadable: false,
readableListening: false,
resumeScheduled: false,
errorEmitted: false,
emitClose: true,
autoDestroy: false,
destroyed: false,
defaultEncoding: 'utf8',
awaitDrainWriters: null,
multiAwaitDrain: false,
readingMore: true,
decoder: null,
encoding: null,
[Symbol(kPaused)]: false
},
readable: false,
_events: [Object: null prototype] {
end: [Array],
close: [Array],
data: [Function (anonymous)],
error: [Function (anonymous)]
},
_eventsCount: 4,
_maxListeners: undefined,
socket: Socket {
connecting: false,
_hadError: false,
_parent: null,
_host: null,
_readableState: [ReadableState],
readable: true,
_events: [Object: null prototype],
_eventsCount: 6,
_maxListeners: undefined,
_writableState: [WritableState],
writable: false,
allowHalfOpen: false,
_sockname: null,
_pendingData: null,
_pendingEncoding: '',
server: null,
_server: null,
parser: null,
_httpMessage: [ClientRequest],
[Symbol(asyncId)]: 15,
[Symbol(kHandle)]: [TCP],
[Symbol(lastWriteQueueSize)]: 0,
[Symbol(timeout)]: null,
[Symbol(kBuffer)]: null,
[Symbol(kBufferCb)]: null,
[Symbol(kBufferGen)]: null,
[Symbol(kCapture)]: false,
[Symbol(kBytesRead)]: 0,
[Symbol(kBytesWritten)]: 0
},
httpVersionMajor: 1,
httpVersionMinor: 1,
httpVersion: '1.1',
complete: true,
headers: {
'content-type': 'application/json',
'set-cookie': [Array],
vary: 'Cookie',
date: 'Wed, 25 Mar 2020 19:45:04 GMT',
'content-length': '210',
connection: 'close'
},
rawHeaders: [
'Content-Type',
'application/json',
'Set-Cookie',
'csrf_token=pNoPNQF60VWUyvylguSn7bdxG3055r3oC28LZEDfS2A=; Domain=127.0.0.1; Max-Age=31536000; HttpOnly',
'Vary',
'Cookie',
'Date',
'Wed, 25 Mar 2020 19:45:04 GMT',
'Content-Length',
'210',
'Connection',
'close'
],
trailers: {},
rawTrailers: [],
aborted: false,
upgrade: false,
url: '',
method: null,
statusCode: 403,
statusMessage: 'Forbidden',
client: Socket {
connecting: false,
_hadError: false,
_parent: null,
_host: null,
_readableState: [ReadableState],
readable: true,
_events: [Object: null prototype],
_eventsCount: 6,
_maxListeners: undefined,
_writableState: [WritableState],
writable: false,
allowHalfOpen: false,
_sockname: null,
_pendingData: null,
_pendingEncoding: '',
server: null,
_server: null,
parser: null,
_httpMessage: [ClientRequest],
[Symbol(asyncId)]: 15,
[Symbol(kHandle)]: [TCP],
[Symbol(lastWriteQueueSize)]: 0,
[Symbol(timeout)]: null,
[Symbol(kBuffer)]: null,
[Symbol(kBufferCb)]: null,
[Symbol(kBufferGen)]: null,
[Symbol(kCapture)]: false,
[Symbol(kBytesRead)]: 0,
[Symbol(kBytesWritten)]: 0
},
_consuming: false,
_dumped: false,
req: ClientRequest {
_events: [Object: null prototype],
_eventsCount: 5,
_maxListeners: undefined,
outputData: [],
outputSize: 0,
writable: true,
_last: true,
chunkedEncoding: false,
shouldKeepAlive: false,
useChunkedEncodingByDefault: false,
sendDate: false,
_removedConnection: false,
_removedContLen: false,
_removedTE: false,
_contentLength: 0,
_hasBody: true,
_trailer: '',
finished: true,
_headerSent: true,
socket: [Socket],
_header: 'GET /self-service/browser/flows/requests/login?request=0fb8e59d-1d7d-4f54-b509-101540ac7384 HTTP/1.1\r\n' +
'Accept: application/json\r\n' +
'host: 10.0.27.88:4434\r\n' +
'Connection: close\r\n' +
'\r\n',
_onPendingData: [Function: noopPendingOutput],
agent: [Agent],
socketPath: undefined,
method: 'GET',
maxHeaderSize: undefined,
path: '/self-service/browser/flows/requests/login?request=0fb8e59d-1d7d-4f54-b509-101540ac7384',
_ended: true,
res: [Circular *1],
aborted: false,
timeoutCb: null,
upgradeOrConnect: false,
parser: null,
maxHeadersCount: null,
reusedSocket: false,
[Symbol(kCapture)]: false,
[Symbol(kNeedDrain)]: false,
[Symbol(corked)]: 0,
[Symbol(kOutHeaders)]: [Object: null prototype]
},
request: Request {
_events: [Object: null prototype],
_eventsCount: 5,
_maxListeners: undefined,
method: 'GET',
headers: [Object],
uri: [Url],
useQuerystring: false,
callback: [Function (anonymous)],
readable: true,
writable: true,
explicitMethod: true,
_qs: [Querystring],
_auth: [Auth],
_oauth: [OAuth],
_multipart: [Multipart],
_redirect: [Redirect],
_tunnel: [Tunnel],
setHeader: [Function (anonymous)],
hasHeader: [Function (anonymous)],
getHeader: [Function (anonymous)],
removeHeader: [Function (anonymous)],
localAddress: undefined,
pool: {},
dests: [],
__isRequestRequest: true,
_callback: [Function (anonymous)],
proxy: null,
tunnel: false,
setHost: true,
originalCookieHeader: undefined,
_disableCookies: true,
_jar: undefined,
port: '4434',
host: '10.0.27.88',
url: [Url],
path: '/self-service/browser/flows/requests/login?request=0fb8e59d-1d7d-4f54-b509-101540ac7384',
_json: true,
httpModule: [Object],
agentClass: [Function],
agent: [Agent],
_started: true,
href: 'http://10.0.27.88:4434/self-service/browser/flows/requests/login?request=0fb8e59d-1d7d-4f54-b509-101540ac7384',
req: [ClientRequest],
ntick: true,
response: [Circular *1],
originalHost: '10.0.27.88:4434',
originalHostHeaderName: 'host',
responseContent: [Circular *1],
_destdata: true,
_ended: true,
_callbackCalled: true,
[Symbol(kCapture)]: false
},
toJSON: [Function: responseToJSON],
caseless: Caseless { dict: [Object] },
body: { error: [Object] },
[Symbol(kCapture)]: false
},
body: LoginRequest {
active: undefined,
expiresAt: undefined,
id: undefined,
issuedAt: undefined,
methods: undefined,
requestUrl: undefined
},
statusCode: 403,
name: 'HttpError'
}
HttpError: HTTP request failed
at Request._callback (/usr/src/app/node_modules/@oryd/kratos-client/dist/api/adminApi.js:299:40)
at Request.self.callback (/usr/src/app/node_modules/request/request.js:185:22)
at Request.emit (events.js:321:20)
at Request.<anonymous> (/usr/src/app/node_modules/request/request.js:1161:10)
at Request.emit (events.js:321:20)
at IncomingMessage.<anonymous> (/usr/src/app/node_modules/request/request.js:1083:12)
at Object.onceWrapper (events.js:427:28)
at IncomingMessage.emit (events.js:333:22)
at endReadableNT (_stream_readable.js:1220:12)
at processTicksAndRejections (internal/process/task_queues.js:84:21)
GET /auth/login?request=0fb8e59d-1d7d-4f54-b509-101540ac7384 500 10761 - 50.248 ms
GET /index.css 304 - - 1.334 ms
GET /typography.css 304 - - 0.922 ms
GET /auth.css 304 - - 0.991 ms
GET /form.css 304 - - 1.379 ms
Kratos config map (relevant parts):
kubectl get cm/ory-kratos -n ory -o yaml
apiVersion: v1
data:
kratos.yaml: |
# serve controls the configuration for the http(s) daemon
serve:
admin:
port: 4433
host: 0.0.0.0
public:
port: 4434
host: 0.0.0.0
dsn: postgres://postgres:[email protected]:5432/kratos
log:
level: debug
courier:
smtp:
connection_uri: smtp://foo:bar@baz/
urls:
default_return_to: http://127.0.0.1:4435/
mfa_ui: http://127.0.0.1:4435/mfa
login_ui: http://127.0.0.1:4435/auth/login
profile_ui: http://127.0.0.1:4435/profile
verify_ui: http://127.0.0.1:4435/verify
registration_ui: http://127.0.0.1:4435/auth/registration
self:
public: http://127.0.0.1:4433
admin: http://127.0.0.1:4434
error_ui: http://127.0.0.1:4435/error
whitelisted_return_to_domains:
- http://127.0.0.1
And this is the Kratos Pod:
apiVersion: v1
kind: Pod
metadata:
labels:
app: ory-kratos
app.kubernetes.io/instance: ory-kratos
app.kubernetes.io/name: ory-krato
name: ory-kratos-5869584dcf-mq4jx
namespace: ory
spec:
containers:
- args:
- serve
- --dev
- --disable-telemetry
- --config
- /etc/config/kratos.yaml
image: oryd/kratos:v0.1.1
imagePullPolicy: IfNotPresent
livenessProbe:
failureThreshold: 5
httpGet:
path: /health/alive
port: http-admin
scheme: HTTP
initialDelaySeconds: 5
periodSeconds: 30
successThreshold: 1
timeoutSeconds: 5
name: ory-kratos
ports:
- containerPort: 4433
hostPort: 4433
name: http-public
protocol: TCP
- containerPort: 4434
hostPort: 4434
name: http-admin
protocol: TCP
readinessProbe:
failureThreshold: 5
httpGet:
path: /health/ready
port: http-admin
scheme: HTTP
initialDelaySeconds: 5
periodSeconds: 30
successThreshold: 1
timeoutSeconds: 5
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /etc/config
name: ory-kratos-config-volume
readOnly: true
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: ory-kratos-token-gjhb8
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
initContainers:
- command:
- kratos
- migrate
- sql
- -e
- -y
- -c
- /etc/config/kratos.yaml
image: oryd/kratos:v0.1.1
imagePullPolicy: IfNotPresent
name: ory-kratos-init
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /etc/config
name: ory-kratos-config-volume
readOnly: true
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: ory-kratos-token-gjhb8
readOnly: true
nodeName: xxxx.yyy.compute.internal
priority: 0
restartPolicy: Always
schedulerName: default-scheduler
securityContext:
fsGroup: 1000
serviceAccount: ory-kratos
serviceAccountName: ory-kratos
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- configMap:
defaultMode: 420
name: ory-kratos
name: ory-kratos-config-volume
- name: ory-kratos-token-gjhb8
secret:
defaultMode: 420
secretName: ory-kratos-token-gjhb8
And finally the UI node Pod:
apiVersion: v1
kind: Pod
metadata:
labels:
app: ory-kratos
app.kubernetes.io/instance: ory-kratos-selfservice-ui-node
app.kubernetes.io/name: ory-kratos-selfservice-ui-node
name: ory-kratos-selfservice-ui-node-57b9dc8789-n9zf6
namespace: ory
spec:
containers:
- env:
- name: PORT
value: "4435"
- name: JWKS_URL
value: http://ory-hydra.ory.svc.cluster.local:4444/.well-known/jwks.json
- name: KRATOS_ADMIN_URL
value: http://10.0.27.88:4434
- name: KRATOS_PUBLIC_URL
value: http://10.0.27.88:4433
- name: KRATOS_BROWSER_URL
value: http://127.0.0.1:4434
- name: BASE_URL
value: /
image: oryd/kratos-selfservice-ui-node:v0.1.1-alpha.1
imagePullPolicy: IfNotPresent
name: ory-ory-kratos-selfservice-ui-node
ports:
- containerPort: 3000
hostPort: 4435
name: http-public
protocol: TCP
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: ory-kratos-token-gjhb8
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
nodeName: ip-10-0-23-162.eu-north-1.compute.internal
priority: 0
restartPolicy: Always
schedulerName: default-scheduler
securityContext:
fsGroup: 1000
serviceAccount: ory-kratos
serviceAccountName: ory-kratos
terminationGracePeriodSeconds: 30
tolerations:
- effect: NoExecute
key: node.kubernetes.io/not-ready
operator: Exists
tolerationSeconds: 300
- effect: NoExecute
key: node.kubernetes.io/unreachable
operator: Exists
tolerationSeconds: 300
volumes:
- name: ory-kratos-token-gjhb8
secret:
defaultMode: 420
secretName: ory-kratos-token-gjhb8
So, why is there no csrf_token in the request to the self-service UI node, if this is the problem? Configuration issues?
Any help is greatly appreciated.
Lars
