Docker daemon permi...
 
Notifications
Clear all

Docker daemon permissions to access NFS share

10 Posts
2 Users
4 Reactions
43 Views
Posts: 5
Topic starter
(@wuruwhi)
Active Member
Joined: 4 days ago

Hello there,

New to the forum, but not to the YT channel, love the content! 🙂 

I’m rebuilding my home lab and changing how I do things.

I have my NAS setup with a NFS share and a specific user owner of it, with a defined uid/gid (ex:1234). My goal is to restrict access to it to a group of containers and VMs. I also set allowed hostnames and IPs to access it.

I deployed a Ubuntu server and manually created a user with the same uid/gid as the one owning the NFS share.
From Ubuntu, I am able to mount the share, access it, write etc… when logged in with the uid=1234 user. So far so good. Entry has been added to fstab.
With that same user, I installed Docker and then edited the « /lib/systemd/system/docker.service » file to add the « —data-root /MyNFS/MountPoint/docker » argument. Copied over the files that the documentation said to move.

However, after rebooting it fails to start Docker and the journalctl file mentions the following error: « could not create or set daemon root permissions: /MyNFS/MountPoint/docker: chown /MyNFS/MountPoint/docker: operation not permitted ».
I tried adding the user to the docker group, or root to the 1234 gid group, but none of that worked.

I thought that running Docker as the 1234 uid user, it would be able to access and write in the share.
What do I need to do for Docker to start and be able to write in the share, but still limiting access to the nfs share to uid/gid 1234?

Cheers 🙂 

Topic Tags
9 Replies
Posts: 5
Topic starter
(@wuruwhi)
Active Member
Joined: 4 days ago

I also tried to modify the « /etc/docker/daemon.json » instead of the one in my previous post, but it still throws an error « unable to configure the Docker daemon with file /etc/docker/daemon.json: the following directives are specified both as a flag and in the configuration file: data-root (from flag: /MyNFS/MountPoint/docker, from file: /MyNFS/MountPoint/docker) »

So I am not sure what should be my next steps.

Reply
Brandon Lee
Posts: 416
Admin
(@brandon-lee)
Member
Joined: 14 years ago

@wuruwhi welcome to the forums! Glad to have you here and hopefully we can work through the NFS issue with Docker. Let me take a look at the information in the posts you have made and I will get back to you with my thoughts if that sounds good and some troubleshooting steps 👍 

Reply
Brandon Lee
Posts: 416
Admin
(@brandon-lee)
Member
Joined: 14 years ago

Ok, so a few thoughts here.

Docker needs full control of the data-root directory. So it might need to perform operations that are conflicting with your NFS permissions or the user-level permissions.

Make sure root_squash is not enabled for your NFS export. If it is, change it to no_root_squash temporarily to test if this resolves the issue:

/MyNFS/MountPoint 192.168.x.x(rw,sync,no_root_squash), then restart the NFS server after making your changes. Is this running on a Linux server? If so, you can do something like sudo exportfs -ra
after making your changes.

On the NFS server, make sure your directory /MyNFS/MountPoint/docker is owned by uid=1234 and

gid=1234.
Run the following commands on the NFS server:

sudo chown -R 1234:1234 /MyNFS/MountPoint/docker
sudo chmod -R 775 /MyNFS/MountPoint/docker

Also, make sure your share is mounted with the right options for Docker. Edit your /etc/fstab or mount command to use the following:

nfs defaults,rw,sync 0 0

Make sure your Docker daemon runs with the same user uid=1234. You can do this by modifying your docker.service file to include the directive:

[Service]
user=1234

The last error that you posted there looks like there is conflicting docker configurations. You will want to use either systemd configuration file or the daemon.json file but not both. If you are using systemd configuration, make sure no data-root directive is in your daemon.json file. If you are using the daemon.json file, remove the --data-root flag from your systemd service file if that makes sense.

Once you have made your changes, try to restart the docker service:

sudo systemctl daemon-reload
sudo systemctl restart docker
Reply
Brandon Lee
Posts: 416
Admin
(@brandon-lee)
Member
Joined: 14 years ago

Also, just a simpler approach to think about. When i am using external storage like NFS, or any other, you don't have to change all your docker storage to use the external storage, but simply bind mount your containers that you want to have stored on your external storage bind mounted to that specific directory. This will simplify your challenges with trying to change the entire Docker data root to be your NFS share. 

So, for example:

version: "3.8"

services:
  app:
    image: nginx:latest
    container_name: container-I-want-on-nfs-storage
    ports:
      - "8080:80"
    volumes:
      - /mnt/nfs_share:/usr/share/nginx/html:rw
    restart: always

The bind mount will make sure the data for your container is stored on NFS while the data-root for docker itself remains the default. This way you can protect your data that you want for those specific containers and not change Docker's data root.

 

Reply
Page 1 / 2