Skip to content

Locale (Internationalization) bug for reading floating point values #514

Closed
@steinlec

Description

@steinlec

As the JSON specification defines the floating point format independent of any internationalization by simply using a point for the decimal point separator, while the json_reader uses the locale dependent scanf(...) function, there is obviously a mismatch in the value Interpretation. However it is immediatly clear that this circumstance has only effect for languages that do have a decimal point separator different from point.

Since the often mentioned workaround to temporarily modify the locale is evidently not practicable for multithreaded programs that uses GUI toolkits, I had a look to the fix that is reported for Ubuntu. But the corresponding patch is removed from the code (Compare json_tool.h: We had a sophisticated way, but it did not work in WinCE. @see #9). So I had a further look to the function static inline void fixNumericLocale(char* begin, char* end) in the same file, and would propose the following modification:

--- json_reader.cpp.orig    2016-03-25 12:09:14.000000000 +0100
+++ json_reader.cpp.new 2016-08-08 17:42:40.921538600 +0200
@@ -1619,6 +1619,7 @@
     Char buffer[bufferSize + 1];
     memcpy(buffer, token.start_, ulength);
     buffer[length] = 0;
+    fixNumericLocaleInput(buffer, buffer + length);
     count = sscanf(buffer, format, &value);
   } else {
     JSONCPP_STRING buffer(token.start_, token.end_);
--- json_tool.h.orig    2016-03-25 12:09:14.000000000 +0100
+++ json_tool.h.new 2016-08-09 08:55:09.156182800 +0200
@@ -6,6 +6,8 @@
 #ifndef LIB_JSONCPP_JSON_TOOL_H_INCLUDED
 #define LIB_JSONCPP_JSON_TOOL_H_INCLUDED

+#include <clocale>
+
 /* This header provides common string manipulation support, such as UTF-8,
  * portable conversion from/to string...
  *
@@ -82,6 +84,18 @@
   }
 }

+static inline void fixNumericLocaleInput(char* begin, char* end) {
+  struct lconv* lc = localeconv();
+  if ((lc != NULL) && (*(lc->decimal_point) != '.')) {
+    while (begin < end) {
+      if (*begin == '.') {
+        *begin = *(lc->decimal_point);
+      }
+      ++begin;
+    }
+  }
+}
+
 } // namespace Json {

 #endif // LIB_JSONCPP_JSON_TOOL_H_INCLUDED

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions