Mounting a NVME disk on AWS EC2

user_6434448 0
Viewed 56872

So I created i3.large with NVME disk on each nodes, here was my process :

  1. lsblk -> nvme0n1 (check if nvme isn't yet mounted)
  2. sudo mkfs.ext4 -E nodiscard /dev/nvme0n1
  3. sudo mount -o discard /dev/nvme0n1 /mnt/my-data
  4. /dev/nvme0n1 /mnt/my-data ext4 defaults,nofail,discard 0 2
  5. sudo mount -a (check if everything is OK)
  6. sudo reboot

So all of this works, I can connect back to the instance. I have 500 GiB on my new partition.

But after I stop and restart the EC2 machines, some of them randomly became inaccessible (AWS warning only 1/2 test status checked)

When I watch the logs of why it is inaccessible it tells me, it's about the nvme partition (but I did sudo mount -a to check if this was ok, so I don't understand)

I don't have the AWS logs exactly, but I got some lines of it :

Bad magic number in super-block while trying to open

then the superblock is corrupt, and you might try running e2fsck with an alternate superblock:

/dev/fd/9: line 2: plymouth: command not found

mount amazon-web-services amazon-ec2
4 Answers

I have been using "c5" type instances since almost a month, mostly "c5d.4xlarge" with nvme drives. So, here's what has worked for me on Ubuntu instances:

first get the location nvme drive is located at:

lsblk

mine was always mounted at nvme1n1. Then check if it is an empty volume and doens't has any file system, (it mostly doesn't, unless you are remounting). the output should be /dev/nvme1n1: data for empty drives:

sudo file -s /dev/nvme1n1

Then do this to format(if from last step you learned that your drive had file system and isn't an empty drive. skip this and go to next step):

sudo mkfs -t xfs /dev/nvme1n1

Then create a folder in current directory and mount the nvme drive:

sudo mkdir /data
sudo mount /dev/nvme1n1 /data

you can now even check it's existence by running:

df -h
user_118149960

You may find useful new EC2 instance family equipped with local NVMe storage: C5d.

See announcement blog post: https://aws.amazon.com/blogs/aws/ec2-instance-update-c5-instances-with-local-nvme-storage-c5d/

enter image description here

Some excerpts from the blog post:

  • You don’t have to specify a block device mapping in your AMI or during the instance launch; the local storage will show up as one or more devices (/dev/nvme*1 on Linux) after the guest operating system has booted.
  • Other than the addition of local storage, the C5 and C5d share the same specs.
  • You can use any AMI that includes drivers for the Elastic Network Adapter (ENA) and NVMe
  • Each local NVMe device is hardware encrypted using the XTS-AES-256 block cipher and a unique key.
  • Local NVMe devices have the same lifetime as the instance they are attached to and do not stick around after the instance has been stopped or terminated.
user_27273170
Related