Some OpenShift Container Platform Learnings
Arbitrary User IDs
One of the first unexpected things I learned when trying to run an existing Docker image on OCP, is that, by default, OCP runs containers as an unpredictable, non-root user.
By default, OpenShift Container Platform runs containers using an arbitrarily assigned user ID. This provides additional security against processes escaping the container due to a container engine vulnerability and thereby achieving escalated permissions on the host node.
This causes problems either for images that expect to run as root, or potentially for ones that expect to run as a specific non-root user, like WebSphere Liberty.
And actually, Liberty itself didn’t have a problem, but some other custom things I’d done to our Liberty - installing Contrast Security into the image - did have a problem that required the proposed workaround technique.
RUN mkdir /home/default/contrast && \
chown -R 1001:0 /home/default/contrast && \
chmod -R g=u /home/default/contrast
Run as root
The next image I tried that hits this same problem right “out of the box” is the one for the Shaarli bookmarking app. And while I have an Issue open to try to get it able to run under OCP, thus far my attempts at the above technique have been unsuccessful.
So in the meantime, I explored how to bypass OCP’s default and explicitly allow this container to run as root. This reference in the older OCP 3.11 documentation gives the relevant commands for that version. And while I couldn’t find the equivalent in the documentation for the 4.5 version that we’re running, a bit of extrapolation and the image did start successfully:
$ oc adm policy add-scc-to-user anyuid system:serviceaccount:shaarli:default
Debugging tips
In the process of debugging, I found Executing commands inside a container as the OCP way to override a container entrypoint. Although for this particular problem, the suggested technique of changing the entry point to a shell seems like it might not make sense in an OCP environment. I know it didn’t actually work here - the container failed to start - but I didn’t pursue further.
I also found How do I debug an application that fails to start up?:
$ oc debug deployment/shaarli
Starting pod/shaarli-debug, command was: /bin/sh
But regrettably, that puts you in the pod as root
(which seems an odd default, given that the pods don’t run as root
by default).
Then in my case, I tried:
$ oc debug --as-root=false deployment/shaarli
But regrettably that never gave me a shell, it just hung there until the oc
command timed out and disconnected. So I don’t know whether that will prove to be a useful debugging technique for this particular problem in general.