WSL2 - Linux relative symlinks broken when accessed from windows only for the \\wsl$\ mount-point

Viewed 6729

Problem

I am really struggling with relative symlinks on wsl2 when they are created in the linux-native filesystem and I want to access the files via the share point \\wsl$\distro-name\whatever - They are simply broken.

Environment

I have wsl2 activated in my Windows10. I have an Ubuntu-20.04:

Ubuntu in WSL2

Impact in my coding workflow

Broken symlinks forbid me to seamlessly "execute in wsl2" while "edit from an IDE in Windows".

Real use case (but not limited to): Developing two intelaced projects: A repo with an application and another repo that lives aside with a library. The application symlinks the library:

  • Main program in /files/repos/my-nice-app
  • Library also in /files/repos/my-nice-lib
  • my-nice-app/libs/my-nice-lib is a symlink to ../../my-nice-lib
  • Intelligent IDE in windows, operating on the application opening \\wsl$\Ubuntu-20.04\files\repos\my-nice-app

With this setup, the location \\wsl$\Ubuntu-20.04\files\repos\my-nice-app\libs\my-nice-lib is expected to map to \\wsl$\Ubuntu-20.04\files\repos\my-nice-lib.

But it does not work. All the code-completion in the IDE is messed up, because the symlink does not de-map well and the IDE can't read the classes and definitions of the library.

How to reproduce a working example

Working example. Step 1 - Preparation

Whenever I create a symlink from the linux in the NTFS filesystem it is properly decoded in windows.

Same the oposite side: If I create the link from windows (both with CMD and mklink or Powershell with New-Item) they are properly decoded in linux.

Imagine this scenario:

  1. I have this dir: /mnt/c/tmp which corresponds to C:\tmp.

  2. I put some contents into a file original.txt. I use the linux bash for that. Create content on NTFS

  3. From the linux, I do a relative symlink linux.txt pointing to original.txt. Create link from linux on NTFS

  4. I then do it from windows. From a CMD with the mklink command: Create link from CMD on NTFS

  5. I can even do the symlink in the windows-side with the New-Item command from an elevated powershell Create link from PowerShell on NTFS

Up to here I should have one file original.txt and three links linux.txt, cmd.txt and powershell.txt

Working example. Step 2 - Listing symlinks

Success: I do see all them listed in each of the 3 shells: linux, cmd and powershell:

Listing in NTFS

Here in Linux (1 in the image) we see they are symlinks, as well as from the CMD (2 in the image) and in the powershell (3 in the image).

Both Linux and CMD also report the "demapping" (4 in the image). As cmd.txt and linux.txt are both relative symlinks there's no magic to do behind, just understand that they are links and done.

Powershell for some reason which I don't care for this question, promoted the relative symlink to an absolute one. This shows up a very interesting effect:

Somebody behind the scenes must be doing some kind of translation work, which is being done well in this case (5 in the image): While from linux powershell.txt it is pointing to a path starting with /mnt/c/... the windows interpreter is seeing it as pointing to C:\....

Working example. Step 3 - Accessing the contents via symlinks

Now time to see if I can cat (type in windows) the contents of all of them...

Accessing in NTFS

No explanation here needed. All 9 combinations (3 creation methods x 3 consumption methods) including the relative and absolute links, all work perfectly.

Now time for the rule-breakers...

How to reproduce a failing example

I'll do the very exact same process but instead of doing it on /mnt/c/tmp I'll do it on /tmp and in windows, instead of accessing it from C:\tmp I'll access it from \\wsl$\Ubuntu-20.04\tmp.

Let's start...

Failing example. Step 1 - preparation

I start by the linux. Navigating to /tmp and creating some dummy content on the WSL2 filesystem. I continue by doing the symlink.

Creating content wsl2

When I try to go there with the CMD I really can't because it complains of being an UNC path:

Navigating UNC on CMD

I'll change my strategy and I'll do a net mount to have a drive letter, see if CMD likes it more. I'll use W: for the WSL2 filesystem. In the image: 1 = I create it, 2 = I check it is created, 3 = I navigate into tmp on the WSL2.

Creating W:

But now... oh surprise!!! When I try to do a symlink from the CMD... it denies the access:

Linking from CMD on WSL2

Let's try with an elevated PowerShell...

In this image I can see I can properly navigate to the UNC path (1 in the image) but when trying to create the link... boom... 2 in the image: "Symbolic links are not compatible with the specified path":

Linking from PowerShell on WSL2

So there's only ONE way to create the symlinks in the WSL2: From inside the linux. Let's see how can we list it and access it.

Failing example. Step 2 - listing + accessing

Starting off, linux can see linux links (of course):

Linux accessing WSL2

But when moving to CMD, the listing shows "JUNCTION" instead of "SYMLINK" as it showed it on the NTFS and additionally when trying to access it, it breaks:

CMD accessing WSL2

Finally when moving into Powershell the behaviour is similar: It it sees "it is there" but the contents can't be accessed:

PowerShell accessing WSL2

Final considerations

  • I'm even not asking for absolute path conversion (as demonstrated that in the NTFS works). I'm just happy with relative links
  • I've done this with files. But it also fails with directories.

Soooooooooo Question

How can I have a properly working symlink on WSL2 working well both in the linux side and the windows side?

If it's a bug, what module is it? The kernel? The WSL itself? The P9 protocol? I'd be more than happy to contribute but I'd even don't know what project should I contribute to.

Investigation done so far

I have deepely read in full all of these:

and many more, but still no luck.

2 Answers

It look like a file permission issue:
When you write in /mnt/c/tmp, you write in the Windows filesystem.
While doing the same in /tmp, you write in the Linux filesystem.
The Linux files are accessed through a Plan9 Server (9P2000L protocol) running on the Windows host with the help of a Unix socket.
The Linux filesystem is accessible through \\wsl$\distribution-name.
I can suggest the video which explain the access of Linux files in Windows.
Explanation of the access of Linux filesystem with Windows 10
9P2000L protocol

It seems the symlink issue is an open issue.

I'm not aware of a way to make use of relative symbolic links in order to do what you want. I suspect there is no sane and reasonable solution to your question as stated besides perhaps fixing the bug linked in the other answer. I recommend taking a different approach altogether that addresses your underlying need of executing in WSL2 and editing in Windows. Our team has the same underlying need, and I have invested considerable (but certainly not exhaustive) effort experimenting with a variety of strategies. My personal recommendation is to use Git to clone your clone, but if anyone has any better ideas, I am all ears!

  • Launch your WSL2 Linux distribution.
  • Optionally, secondary click on the top of the window, select Properties, check "Use Ctrl+Shift+C/V as Copy/Paste," and select OK.
  • Run git clone git@github.com:<owner>/<repository> to clone this repository to your Linux file system.
  • Clone your clone to the Windows file system: git clone ~/source/repos/<repository> /mnt/<drive>/Users/<username>/.../<repository> --branch main. This allows you to push to and pull from the clone in the Linux file system.

Another option would be to run rsync --archive --delete-after <path to Windows copy> <path to Linux copy> from your WSL2 Linux distribution. This way you can efficiently copy any changes made to your project in the Windows file system to the Linux file system before execution. You could even configure a cron job to automatically run rsync in the background periodically if the Linux side is read-only.

I don't recommend copying files between file systems using Windows-side tools like PowerShell (or even Git Bash), because the Linux file system has more granular permissions that Windows tools have a tendency to mangle. The one exception is Git, because Git itself handles these sorts of cross-platform compatibility issues regardless of how it is run. You should, however, configure your .gitattributes appropriately to manage differences in line endings automatically.

On October 5th, 2021, Microsoft released Windows 11 with Linux GUI app support. Thus, instead of doing any of this, if it's acceptable, you could simply use an IDE for Linux, such as VSCode. (This doesn't solve my team's problem unfortunately, because we execute on both the Linux and Windows sides.)

Related