Discussion:
[SM-Discuss] LC usage in cast
Treeve Jelbert
2017-10-14 12:21:03 UTC
Permalink
when I cast certain spells I get the following

Warning: You are using 'ANSI_X3.4-1968' which is not a
Unicode-compatible locale.
You might see errors if you use UTF-8 strings as filenames, as strings,
or as file contents.
Please switch to a UTF-8 locale for your platform.

My environment is
# env |grep LC
LC_ALL=en_GB.UTF-8
LC_TIME=


I see that cast resets LC_ALL

# HACK for bug 2910 and 10546
if [[ -n ${saved_lc_all} ]] ; then
export LC_ALL=$saved_lc_all
else
unset LC_ALL
fi

If I disable this code, the warning disappears.

Is that hack still needed, or can we haz?
Pavel Vinogradov
2017-10-15 01:47:06 UTC
Permalink
Meson bug? If so, disabling the code in cast didn't work in my case (cast libinput). Strange, no recent changes in meson, python or ninja...
Post by Treeve Jelbert
when I cast certain spells I get the following
Warning: You are using 'ANSI_X3.4-1968' which is not a
Unicode-compatible locale.
You might see errors if you use UTF-8 strings as filenames, as
strings, or as file contents.
Please switch to a UTF-8 locale for your platform.
My environment is
# env |grep LC
LC_ALL=en_GB.UTF-8
LC_TIME=
I see that cast resets LC_ALL
# HACK for bug 2910 and 10546
if [[ -n ${saved_lc_all} ]] ; then
export LC_ALL=$saved_lc_all
else
unset LC_ALL
fi
If I disable this code, the warning disappears.
Is that hack still needed, or can we haz?
_______________________________________________
SM-Discuss mailing list
https://lists.ibiblio.org/mailman/listinfo/sm-discuss
--
Sincerely,
Pavel Vinogradov
Eric Sandall
2018-03-06 22:15:44 UTC
Permalink
Post by Treeve Jelbert
when I cast certain spells I get the following
Warning: You are using 'ANSI_X3.4-1968' which is not a
Unicode-compatible locale.
You might see errors if you use UTF-8 strings as filenames, as
strings, or as file contents.
Please switch to a UTF-8 locale for your platform.
My environment is
# env |grep LC
LC_ALL=en_GB.UTF-8
LC_TIME=
I see that cast resets LC_ALL
  # HACK for bug 2910 and 10546
  if [[ -n ${saved_lc_all} ]] ; then
    export LC_ALL=$saved_lc_all
  else
    unset LC_ALL
  fi
If I disable this code, the warning disappears.
Is that hack still needed, or can we haz?
I'm also running into this (devel Sorcery, test grimoire) and I don't
have LC_* set:

# env | grep LC
#


Error when casting xorgproto

FAILED: include/X11/Xpoll.h
/usr/bin/python3 /usr/bin/meson --internal exe
/usr/src/xorgproto-2018.4/builddir/meson-private/meson_exe_sed_c3bd32f04cf001aaa65427d8050bff16c6734f5b.dat
Warning: You are using 'ANSI_X3.4-1968' which is not a
Unicode-compatible locale.
You might see errors if you use UTF-8 strings as filenames, as
strings, or as file contents.
Please switch to a UTF-8 locale for your platform.
Traceback (most recent call last):
  File "/usr/bin/meson", line 37, in <module>
    sys.exit(main())
  File "/usr/bin/meson", line 34, in main
    return mesonmain.run(sys.argv[1:], launcher)
  File "/usr/lib/python3.6/site-packages/mesonbuild/mesonmain.py",
line 322, in run
    sys.exit(run_script_command(args[1:]))
  File "/usr/lib/python3.6/site-packages/mesonbuild/mesonmain.py",
line 278, in run_script_command
    return cmdfunc(cmdargs)
  File
"/usr/lib/python3.6/site-packages/mesonbuild/scripts/meson_exe.py",
line 76, in run
    return run_exe(exe)
  File
"/usr/lib/python3.6/site-packages/mesonbuild/scripts/meson_exe.py",
line 59, in run_exe
    p, stdout, stderr = Popen_safe(cmd + exe.cmd_args,
env=child_env, cwd=exe.workdir)
  File "/usr/lib/python3.6/site-packages/mesonbuild/mesonlib.py",
line 642, in Popen_safe
    o, e = p.communicate(write)
  File "/usr/lib/python3.6/subprocess.py", line 843, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
  File "/usr/lib/python3.6/subprocess.py", line 1554, in _communicate
    self.stdout.errors)
  File "/usr/lib/python3.6/subprocess.py", line 740, in
_translate_newlines
    data = data.decode(encoding, errors)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position
1160: ordinal not in range(128)
ninja: build stopped: subcommand failed.


If I remove this code from /usr/sbin/cast:269-273, casting xorgproto
(which uses meson) still fails with the same error. Earlier in
/usr/sbin/cast, lines 259-260, we have:
  # HACK for bug 2910 and 10546
  local saved_lc_all=$LC_ALL
  export LC_ALL=C

If I change LC_ALL to "en_US.UTF-8" in /usr/sbin/cast instead of "C",
casting xorgproto with meson works, even if I leave in the other 2910
hack on lines 269-273.

So the issue is that cast is overriding LC_ALL to "C", which meson does
not like because :reasons:, even if others have a sane value.

Sorcery should only set LC_ALL if it's unset, IMO, and we may need to
pick a language, e.g. "en_US.UTF-8" to provide as a fallback. ;)

-sandalle
Ismael Luceno
2018-03-10 22:22:31 UTC
Permalink
Post by Eric Sandall
Post by Treeve Jelbert
when I cast certain spells I get the following
Warning: You are using 'ANSI_X3.4-1968' which is not a
Unicode-compatible locale.
You might see errors if you use UTF-8 strings as filenames, as
strings, or as file contents.
Please switch to a UTF-8 locale for your platform.
My environment is
# env |grep LC
LC_ALL=en_GB.UTF-8
LC_TIME=
I see that cast resets LC_ALL
  # HACK for bug 2910 and 10546
  if [[ -n ${saved_lc_all} ]] ; then
    export LC_ALL=$saved_lc_all
  else
    unset LC_ALL
  fi
If I disable this code, the warning disappears.
Is that hack still needed, or can we haz?
I'm also running into this (devel Sorcery, test grimoire) and I don't
# env | grep LC
#
Error when casting xorgproto
FAILED: include/X11/Xpoll.h
/usr/bin/python3 /usr/bin/meson --internal exe
/usr/src/xorgproto-2018.4/builddir/meson-private/meson_exe_sed_c3bd32f04cf001aaa65427d8050bff16c6734f5b.dat
Warning: You are using 'ANSI_X3.4-1968' which is not a
Unicode-compatible locale.
You might see errors if you use UTF-8 strings as filenames, as
strings, or as file contents.
Please switch to a UTF-8 locale for your platform.
  File "/usr/bin/meson", line 37, in <module>
    sys.exit(main())
  File "/usr/bin/meson", line 34, in main
    return mesonmain.run(sys.argv[1:], launcher)
  File "/usr/lib/python3.6/site-packages/mesonbuild/mesonmain.py",
line 322, in run
    sys.exit(run_script_command(args[1:]))
  File "/usr/lib/python3.6/site-packages/mesonbuild/mesonmain.py",
line 278, in run_script_command
    return cmdfunc(cmdargs)
  File
"/usr/lib/python3.6/site-packages/mesonbuild/scripts/meson_exe.py",
line 76, in run
    return run_exe(exe)
  File
"/usr/lib/python3.6/site-packages/mesonbuild/scripts/meson_exe.py",
line 59, in run_exe
    p, stdout, stderr = Popen_safe(cmd + exe.cmd_args,
env=child_env, cwd=exe.workdir)
  File "/usr/lib/python3.6/site-packages/mesonbuild/mesonlib.py",
line 642, in Popen_safe
    o, e = p.communicate(write)
  File "/usr/lib/python3.6/subprocess.py", line 843, in communicate
    stdout, stderr = self._communicate(input, endtime, timeout)
  File "/usr/lib/python3.6/subprocess.py", line 1554, in _communicate
    self.stdout.errors)
  File "/usr/lib/python3.6/subprocess.py", line 740, in
_translate_newlines
    data = data.decode(encoding, errors)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position
1160: ordinal not in range(128)
ninja: build stopped: subcommand failed.
If I remove this code from /usr/sbin/cast:269-273, casting xorgproto
(which uses meson) still fails with the same error. Earlier in
  # HACK for bug 2910 and 10546
  local saved_lc_all=$LC_ALL
  export LC_ALL=C
If I change LC_ALL to "en_US.UTF-8" in /usr/sbin/cast instead of "C",
casting xorgproto with meson works, even if I leave in the other 2910
hack on lines 269-273.
So the issue is that cast is overriding LC_ALL to "C", which meson does
not like because :reasons:, even if others have a sane value.
Sorcery should only set LC_ALL if it's unset, IMO, and we may need to
pick a language, e.g. "en_US.UTF-8" to provide as a fallback. ;)
LC_ALL=C.UTF-8 shuold work ;-).
Ismael Luceno
2018-03-10 22:50:59 UTC
Permalink
<...>
Post by Ismael Luceno
Post by Eric Sandall
So the issue is that cast is overriding LC_ALL to "C", which meson does
not like because :reasons:, even if others have a sane value.
Sorcery should only set LC_ALL if it's unset, IMO, and we may need to
pick a language, e.g. "en_US.UTF-8" to provide as a fallback. ;)
LC_ALL=C.UTF-8 shuold work ;-).
I just noticed we don't provide C.UTF-8 :-(. It can be generated with:
# localedef -f UTF-8 -i POSIX C.UTF-8

We should add it to the glibc spell, I guess.

Loading...