Skip to content

lualatex.fmt not portable between 32-bit/64-bit machines #775

@kberry

Description

@kberry

Evidently lualatex.fmt cannot be shared between 32-bit and 64-bit systems. All other fmts, including xelatex.fmt, pdflatex.fmt, and the plain luahbtex.fmt, are sharable. Of course the fmts are created by the exact same version of luahbtex (current TL svn).

I get the same mesage with either creating lualatex.fmt on 32-bit and reading on 64-bit, or the other way around. (The fmts can be read fine on the machine where they are created.) I run this command in the TL
Build/source/Work/texk/web2c/ directory (I'll explain trytexenv below):

trytexenv ./luahbtex -fmt ~/lualatex.fmt \\stop

where ~/lualatex.fmt has been copied in from the other machine,
and get this error:

This is LuaHBTeX, Version 1.15.0 (TeX Live 2022)                                
 restricted system commands enabled.                                            
LaTeX2e <2021-11-15> patch level 1                                              
L3 programming layer <2021-11-22>...live/karl/Master/texmf-dist/tex/latex/l3ker
nel/expl3.lua:244: bad bytecode register                                        
stack traceback:                                                                
        [C]: in function 'lua.getbytecode'                                      
        ...live/karl/Master/texmf-dist/tex/latex/l3kernel/expl3.lua:244: in main chunk                                                                         
                                                                                
        [C]: in function 'require'                                              
        [\directlua]:1: in main chunk.                                          
\lua_now:e #1->\__lua_now:n {#1}                                                
                                                                                
<*>                                                                             
 \stop                                                                          
?                            

Akira reports that he gets the same message with his Windows binaries in the same cross-architecture situation.

Above I said "all" other fmts work, but in truth there is one other that doesn't: optex. I noticed this last year, and asked Petr about it. I don't know if it is related to whatever is going on with expl3.lua , but FWIW, he wrote (sorry for the poor line breaks, looked fine in my editor):

Date: Tue, 20 Jul 2021 20:02:02 +0200
From: Petr Olsak
Subject: Re: optex.fmt on BigEndian machine?

[...] The idea used in OpTeX (to save a lua code as "bytecode" into the
format file and then use it when the format is used) was borrowed from
ConTeX. So, Context cannot be shared between architectures too.

The lua manual (man luac) says:

Precompiled chunks are not portable across different architectures.

Of course, I can change this concept and read lua files at \everyjob
(not converted to bytecode and not saved to the format) but I prefer to
keep the bytecode saved to the format: it seems to be more compact and
the speed is better. There is very small number of users they want to
share OpTeX between architectures. I hope, they can create two versions
of formats if they want to use OpTeX at two different architectures.

M. Vlasak suggests another solution: use bytecode if it can be
succesfully loaded else read the .lua files again:
olsak/OpTeX@dbf5b6e
but I don't plan to use this code because the code can be usable for
almost zero users.

I expect that it's also the case for lualatex that "almost zero users" will notice if lualatex.fmt written on 64-bit fails to read on 32-bit and vice versa. It's pretty hard even to find such a pair of machines nowadays. However, if it'd ever be a problem it would surely be with latex where it got noticed. So, I wanted to report it to you.

If you want access to a 32-bit machine, I expect Nelson would be happy to provide. That's where I am testing this stuff.

wdyt? --thanks, karl.

Appendix. The trytexenv script I used above is this:

#!/bin/sh
# $Id: trytexenv,v 1.3 2022/02/03 18:26:23 karl Exp $
# envvars to run from build tree.
TEXMFROOT=$tm; export TEXMFROOT
TEXMFCNF=$TEXMFROOT/texmf-dist/web2c; export TEXMFCNF
LUAINPUTS=$TEXMFROOT/texmf-dist/tex//:; export LUAINPUTS
exec "$@"

where $tm is an envvar I define outside the script to be the "Master" tree for whatever system I'm working on. The actual /Master subdir from an svn checkout, etc.

These environment settings are needed because when running from the build dir, of course the usual search based on $0 and SELFAUTO* cannot find texmf.cnf, and thus nothing else is defined.

As for LUAINPUTS, which seems like it should not be necessary, I think it needs to be defined explicitly because of the different binary name. I can get different errors if I change the search order around. I didn't investigate in depth. Without defining it at all, the file lualatexquotejobname.lua can't be loaded, so the fmt reading aborts early.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions