Is this warning still relevant, and if so, will it not have any impact here?

Is this warning still relevant, and if so, will it not have any impact here?

I don't think there has ever been any formal decision about making archives non-experimental, but I think that the functionality has been used in enough places (especially escript archives) that it will now be impossible to remove. I'll check and see if we can remove that warning.

I think that the functionality has been used in enough places (especially escript archives) that it will now be impossible to remove

To provide some data points, Elixir unzips .ez archives on install, so we don't need runtime support for .ez, but we do use escripts archives extensively. rebar3 is also distributed as an escript archive, right?

rebar3 is also distributed as an escript archive, right?

yes, as is erlang_ls and I'm sure many more. If only we could solve the issue with embedding .so/.dll files...

RabbitMQ uses archives for its plugins as well. It would be great to solve the .so problem, but in the meantime I believe we just unzip (or don't zip) those. Considering how widely used they are it would be a good idea to make archives non-experimental and to fix the few remaining edge cases with them. I can probably help with that at some point.

Hey guys, just a bystander here intrigued by the discussion.
If I understand correctly, the .so/.dll issue is that you can't use dlopen and the likes on a memory buffer containing the native code because the syscall takes a path on the disk... Did I get that right?
That's a problem that raises with Python archives as well, py2exe handles this pretty successfully for Windows, might be worth looking into. Not sure how they fare with the unix case though.
For recent-enough linux systems memfd_create can be used to get a cheap memory backed fd, which can be dlopened via /proc/self/fd/%d. Hackish, but should work.

If I understand correctly, the .so/.dll issue is that you can't use dlopen and the likes on a memory buffer containing the native code because the syscall takes a path on the disk... Did I get that right?

Yes, this is the problem. I created a ticket for it here so that we do not continue to highjack this PR.

@garazdawi just one more comment about the whole archives/escripts note. If the goal is to mark them as officially supported, then perhaps we should not merge this PR and instead make it so file:read always looks inside archives? Or is there a high cost associated to this that makes it undesired?

Otherwise I assume rr/1 is not the only function that will fail for archives/escripts. What if a project needs to load other files inside escripts/archives? Such as the doc/*.chunk files or files from priv in general?

@garazdawi just one more comment about the whole archives/escripts note. If the goal is to mark them as officially supported, then perhaps we should not merge this PR

I think that we should still merge this as fixing archives will be a lot of work and will not be done before 24. It will be easy to revert this when/if we implement more support for archives and then we will have a testcase that should work no matter what the implementation is.

instead make it so file:read always looks inside archives? Or is there a high cost associated to this that makes it undesired?

It would mean that we have to do a string:find/2 for each file:open/file:read_file/file:read_file_info, which I don't think should be noticeable in relation to the actual file operations...

I did not dare to modify more serious modules like code or beam_lib fearing unexpected change of behaviour elsewhere. That's why I chose the most shallow and harmless place to quick fix my particular issue. But I agree with Jose that archives/escripts support could be tackled on a much more generic way. Maybe not pure file:read but with an include_archives option to show intent.

Thanks!