Hello guys, as we already discussed it on IRC, I'm creating this BZ so this issue is not forgotten... :) Description of problem: ----------------------- Internationalized domain names exist for quite some time (IDNA2003), although the protocols describing them have evolved in an incompatible way (IDNA2008). These incompatibilities will prevent applications written for IDNA2003 to access certain problematic domain names defined with IDNA2008, e.g., faß.de is translated to domain xn--fa-hia.de with IDNA2008, while in IDNA2003 it is translated to fass.de domain. That not only causes incompatibility problems, but may be used as an attack vector to redirect users to different web sites. The change is about deprecating libidn, which supports IDNA2003, and switch all applications using libidn, to libidn2 2.0.0, which supports IDNA2008. The switch should be transparent as the libidn2 library is API compatible. See instructions at: -------------------- https://libidn.gitlab.io/libidn2/manual/libidn2.html#Converting-from-libidn More info can be found on Fedora wiki: -------------------------------------- https://fedoraproject.org/wiki/Changes/IDNA2008 Solution: --------- Most likely the change to ./configure should be enough in a way that it looks for 'libidn2' by default, and prefers it over the 'libidn' if both of them are found. Best regards, -- Dee'Kej --
Additional information from Nikos: (In reply to David Kaspar [Dee'Kej] from comment #7) > NOTE: AFAIK, upstream is using the libidn for support of UTF-8 passwords in > e.g. locking the PDF files. I'm not aware of them using the libidn for > Domain Names resolutions. Aha, that's good to know. libidn provides stringprep functionality which is what they most probably rely on. In that case it may not be proper to switch to libidn2 as it (IDNA2008) doesn't use stringprep any more (new standards are different and called precis). If ghostscript doesn't use IDNA for DNS resolution it makes sense to close this bug as non applicable.
So, does that mean we can just close this bug?
To be honest, I really am not sure... :-/ Are you sure that you are using the libidn only for UTF-8 passwords support?
We do just use it to normalize password strings for PDFs. My concern is if libidn is deprecated, it may fall into non-maintained mode. If libidn2 supports the stringprep capability, we should use it.
> My concern is if libidn is deprecated, it may fall into non-maintained mode. If > libidn2 supports the stringprep capability, we should use it. I'm affraid that the new stringprep functionality might be different, and therefore passwords set for PDF with libidn wouldn't no longer work with libidn2. In other words the old passwords for old documents wouldn't work with Ghostscript using libidn2. This could make a lot of people unhappy. I'll ask the coordinator of this change in Fedora if he can provide some more info about this. For now, it might be better to keep this BZ opened.
(In reply to David Kaspar [Dee'Kej] from comment #5) > > My concern is if libidn is deprecated, it may fall into non-maintained mode. If > libidn2 supports the stringprep capability, we should use it. > > I'm affraid that the new stringprep functionality might be different, and > therefore passwords set for PDF with libidn wouldn't no longer work with > libidn2. In other words the old passwords for old documents wouldn't work > with Ghostscript using libidn2. This could make a lot of people unhappy. No, that's not quite the way we use it. (simplistically) We read the password from the command line, and the password from the PDF file, and then compare them. The problem is that (typically) Unicode allows certain constructs to be represented in more than one way, so simply taking the two Unicode strings and comparing them won't work reliably. stringprep will "normalize" the two unicode strings into a form we can reliably compare. The results from stringprep should only ever be used internally, so it shouldn't matter if the normalization from libidn2 differs from libidn.
Reading the libidn2 docs, it not longer seems to include the stringprep() API. As far as I can tell, all the libidn2 APIs (AFAICT) expect already normalized UTF-8. Which raises the question: do we need to consider a different UTF string normalizing solution?
(In reply to Chris Liddell (chrisl) from comment #7) > Reading the libidn2 docs, it not longer seems to include the stringprep() > API. As far as I can tell, all the libidn2 APIs (AFAICT) expect already > normalized UTF-8. > > Which raises the question: do we need to consider a different UTF string > normalizing solution? The 'stringprep()' is completely gone from libidn2. I was told that libidn2 does not expect already normalized UTF-8 by default. It upon callee of libidn2 what they will feed it with. Anyway, another thing I was told is that for now, there exist a function 'gnutls_utf8_password_normalize()' inside GNU TLS library that can be used for this purpose: https://gitlab.com/gnutls/gnutls/blob/master/lib/str-unicode.c#L190 This means that for now, you would have to either link your Ghostscript against gnutls library, or bundle the code for 'gnutls_utf8_password_normalize()' inside Ghostscrit's source code. However, none of these are good solution nor optimal... There's a plan to create a new library, called libprecis, which should contain all the necessary functions for normalizing password, usernames, etc.: https://gitlab.com/libidn/libprecis/tree/master/lib However, the work on it hasn't started yet (only private repo has been created, and I don't have access to it). The libprecis library should be based on several RFCs: https://tools.ietf.org/html/rfc8264 https://tools.ietf.org/html/rfc8265 And the 'stringprep()' (and libprecis) is more discussed here: https://gitlab.com/libidn/libidn2/issues/28
We'll revisit this once a viable stringprep() alternative is available.
(In reply to Chris Liddell (chrisl) from comment #9) > We'll revisit this once a viable stringprep() alternative is available. It seems that upstream (libidn2) something has changed. Chan this be revisited?
(In reply to Michael Osipov from comment #10) > (In reply to Chris Liddell (chrisl) from comment #9) > > We'll revisit this once a viable stringprep() alternative is available. > > It seems that upstream (libidn2) something has changed. Chan this be > revisited? Nothing relevant has changed: https://www.gnu.org/software/libidn/libidn2/manual/libidn2.html#Stringprep-and-libidn2 "Applications requiring the stringprep processing should continue using the original libidn"
(In reply to Chris Liddell (chrisl) from comment #11) > (In reply to Michael Osipov from comment #10) > > (In reply to Chris Liddell (chrisl) from comment #9) > > > We'll revisit this once a viable stringprep() alternative is available. > > > > It seems that upstream (libidn2) something has changed. Chan this be > > revisited? > > Nothing relevant has changed: > > https://www.gnu.org/software/libidn/libidn2/manual/libidn2.html#Stringprep- > and-libidn2 > > "Applications requiring the stringprep processing should continue using the > original libidn" Seems so :-( Although I haven't looked at the code or the explicit description of stringprep maybe https://juliastrings.github.io/utf8proc/ is something worth looking at.
(In reply to Michael Osipov from comment #12) > (In reply to Chris Liddell (chrisl) from comment #11) > > (In reply to Michael Osipov from comment #10) > > > (In reply to Chris Liddell (chrisl) from comment #9) > > > > We'll revisit this once a viable stringprep() alternative is available. > > > > > > It seems that upstream (libidn2) something has changed. Chan this be > > > revisited? > > > > Nothing relevant has changed: > > > > https://www.gnu.org/software/libidn/libidn2/manual/libidn2.html#Stringprep- > > and-libidn2 > > > > "Applications requiring the stringprep processing should continue using the > > original libidn" > > Seems so :-( The real problem is that, for reasons unknown to me, libidn2 is not a replacement for libidn, it implements a quite different thing. Which, to my mind, should warrant a different name, not just a "2" appended to the name. > Although I haven't looked at the code or the explicit description of > stringprep maybe https://juliastrings.github.io/utf8proc/ is something worth > looking at. Possibly. But we have a working solution, one which is recommended by the relevant developers, so it will likely be some time before we have time to look at alternatives.
(In reply to Chris Liddell (chrisl) from comment #13) > (In reply to Michael Osipov from comment #12) > > (In reply to Chris Liddell (chrisl) from comment #11) > > > (In reply to Michael Osipov from comment #10) > > > > (In reply to Chris Liddell (chrisl) from comment #9) > > > > > We'll revisit this once a viable stringprep() alternative is available. > > > > > > > > It seems that upstream (libidn2) something has changed. Chan this be > > > > revisited? > > > > > > Nothing relevant has changed: > > > > > > https://www.gnu.org/software/libidn/libidn2/manual/libidn2.html#Stringprep- > > > and-libidn2 > > > > > > "Applications requiring the stringprep processing should continue using the > > > original libidn" > > > > Seems so :-( > > The real problem is that, for reasons unknown to me, libidn2 is not a > replacement for libidn, it implements a quite different thing. Which, to my > mind, should warrant a different name, not just a "2" appended to the name. Intersection isn't a replacement, unfortunate naming from project devs. > > Although I haven't looked at the code or the explicit description of > > stringprep maybe https://juliastrings.github.io/utf8proc/ is something worth > > looking at. > > Possibly. But we have a working solution, one which is recommended by the > relevant developers, so it will likely be some time before we have time to > look at alternatives. I absolutely agree, why fix if it ain't broken?! Leave as-is, don't waste your time. Thanks for the input.