Path length limit causing missing output

Problems running VASP: crashes, internal errors, "wrong" results.


Moderators: Global Moderator, Moderator

Post Reply
Message
Author
oscarlb
Newbie
Newbie
Posts: 2
Joined: Fri Feb 19, 2021 10:19 am

Path length limit causing missing output

#1 Post by oscarlb » Fri Jan 31, 2025 4:21 pm

It has been known for a long time on the cluster we run vasp that you encounter issues when the path length for the run directory goes beyond 256 characters. After recent experience on different clusters I have narrowed down the behavior and cause, and would like to request a solution for coming versions of VASP.

Specifically, if the absolute path for the OUTCAR file is 259 characters, the calculation will start, but after going into the electronic convergence, new output file paths are capped to the 256 character limit, e.g., causing the remaining OUTCAR output to be written to "OUT" in the same folder. It is worse when output files get capped higher up in the path tree where some files are not supposed to be written or not allowed to be written.

The output files are closed before starting the electronic convergence and reopened using the filename obtained from INQUIRE (I am sure it was needed for flushing or other reasons at the time, but is it still necessary?). See subroutines WFORCE, REOPEN, etc. in lib/diolib.F. This filename is saved in a buffer of size 255 characters, which is usually fine since the proper/gnu behaviour of INQUIRE is to return the relative path of the file. However, intel compiler behavior is to return the absolute path, meaning this filepath capping error is encountered when the path is too long. I realize this is also an unusual situation, but in our production runs, we encode a lot of running information in the folder name for various reasons and therefore encounter this error often.

I see some possible solutions I would like to discuss:

  1. Add a compiler option for the size of this buffer, so one may request the cluster to update the current and coming binaries for the intel compilers.

  2. Make the buffer dynamic, increasing the buffer until full path is captured/buffer is properly terminated.

  3. Add flag/compiler option for skipping the reopening of files if users do not see the need.

  4. Inquire to intel about possible options on their end for uniformity across compilers.

On the use end we could always compile with a different compiler or manually modify the buffer size, but none of these solutions are very sustainable moving between clusters and VASP versions.
I would be very glad if you could come with an official solution to this issue, although I otherwise hope this post can serve as documentation for those running older versions with long paths. Thank you in advance for any guidance and discussion.


alexey.tal
Global Moderator
Global Moderator
Posts: 372
Joined: Mon Sep 13, 2021 12:45 pm

Re: Path length limit causing missing output

#2 Post by alexey.tal » Tue Feb 04, 2025 12:33 pm

Dear oscarlb,

Thank you for a very detailed report and for bringing this issue to our attention. We weren't aware of this problem.

Indeed, you identified the issue correctly and I was able to reproduce this behavior with the Intel compiler.

The reason we still use WFORCE is that FLUSH does not guarantee that the data is actually written out and the data might be written only at the very end when the calculation is finished. NVHPC is one such compiler. So far WFORCE has been the most reliable way to flush.

The most straightforward solution is to increase the buffer size. I believe that 1024 characters would suffice or do you use even longer file paths?
We will release the fix in the upcoming version.

Best wishes,
Alexey


Post Reply