Linux convert xml to json

Linux convert xml to json

Fast & easy library & command line tool for converting XML files to JSON.

Heads up! The project has been split into two projects. See xml-to-json-fast for a low-memory-usage version that has less features.

xml-to-json converts xml to json. It includes a Haskell library and a command-line tool.

xml-to-json ships with two different executables:

1. xml-to-json-fast («fast») uses a lot less memory, but you can’t control the output. Can be used on XML files of any size.
2. xml-to-json («classic») provides some control over json output format, but uses a lot of memory. Suitable for smaller files.

The so-called «fast» version (which uses a lot less memory) has been forked into a separate project, xml-to-json-fast.

The fully featured «classic» xml-to-json provides compact json output that’s designed to be easy to store and process using JSON-based databases, such as mongoDB or CouchDB. In fact, the original motivation for xml-to-json was to store and query a large (

10GB) XML-based dataset, using an off-the-shelf scalable JSON database.

When using «classic» xml-to-json, the input XML must be valid.

Currently the xml-to-json processes XMLs according to lossy rules designed to produce sensibly minimal output. If you need to convert without losing information at all consider something like the XSLT offered by the jsonml project. Unlike jsonml, this tool — xml-to-json — produces json output similar (but not identical) to the xml2json-xslt project.

xml-to-json is implemented in Haskell.

As of this writing, xml-to-json uses hxt with the expat-based hxt-expat parser. The pure Haskell parsers for hxt all seem to have memory issues which hxt-expat doesn’t.

The easy way: use Stack

1. Get Haskell Stack
2. Run: stack install xml-to-json

You may need to install libcurl . For that, follow your platform’s instructions. On debian/ubuntu, you can use: sudo apt-get install libcurl4-openssl-dev .

Can’t use Stack? Use cabal-install

Note for Windows users: Only local files, not URLs, are supported as command line arguments. This is because curl doesn’t compile on my (windows + cygwin) machine out-of-the-box.

To install the release version: Since xml-to-json is implemented in Haskell, «all you need to do» is install the latest cabal-install (you can try the Haskell platform) for your system, and then run:

To install from source: Clone this repository locally, and then (assuming you have Haskell platform installed) run cabal install :

Just run the tool with the filename as a single argument, and direct the stdout to a file or a pipe:

Classic xml-to-json : Advanced Usage

Use the —help option to see the full command line options.

Here’s a (possibly outdated) snapshot of the —help output:

JSON output using default settings:

Formatted for readability (not the actual output):

Note that currently xml-to-json does not retain the order of elements / attributes.

Using the various options you can control various aspects of the output such as:

• At which top-level nodes the conversion starts to work (-t)
• Whether to wrap the output in a top-level JSON array,
• Whether or not to collapse simple string elements, such as the element in the example, into a simple string property (you can specify a regular expression pattern to decide which nodes to collapse)
• And more.

Use the —help option to see a full list of options.

Performance of «classic» xml-to-json

The «classic» xml-to-json cannot operate on large files. However, it is fast when operating on multiple small files. For large XML files, the speed on a core-i5 machine is about 2MB of xml / sec, with a 100MB XML file resulting in a 56MB json output. It took about 10 minutes to process 1GB of xml data. The main performance limit is memory — only one single-threaded process was running since every single large file (tens of megabytes) consumes a lot of memory — about 50 times the size of the file.

A few simple tests have shown this to be at least twice as fast as jsonml’s xlst-based converter (however, the outputs are not similar, as stated above).

Currently the program processes files serially. If run in parallel on many small XML files (

About

Fast & easy command line tool for converting XML files to JSON

Источник

Linux convert xml to json

XML To JSON Converter

This repository contains code for the XML to JSON Converter. This converter is written in Python and will convert one or more XML files into JSON / JSONL files

Converts XML to valid JSON or JSONL Requires only two files to get started. Your XML file and the XSD schema file for that XML file. Multiprocessing enabled to parse XML files concurrently if the XML files are in the same format. Call with -m # option. Uses Python’s iterparse event based methods which enables parsing very large files with low memory requirements. This is very similar to Java’s SAX parser Files are processed in order with the largest files first to optimize overall parsing time Option to write results to either Linux or HDFS folders

Convert a small XML file to a JSON file

Alice Smith 123 Maple Street Mill Valley CA 90952 Robert Smith 8 Oak Avenue Old Town PA 95819 Hurry, my lawn is going wild!

1 148.95 Confirm this is electric

1 39.98 1999-05-21

JSON output (zip looks funny, but blame Microsoft which says zip is a decimal in the XSD file spec )

Convert an entire directory of XML files to JSONL

Also zip output files, parse 3 files concurrently, only extract /PurchaseOrder/items/item elements and incrementally process one XML path at a time to save memory instead of trying to read the entire XML file into memory.

Add additional attributes from other elements

Only attributes from elements found before the xpath can be include

Источник

How to convert XML to JSON in Node.js

Last week, I wrote an article on how to convert a JSON array into a CSV file in a Node.js application. Today, we’ll look at how to parse an XML file or a string to a JSON object.

To perform the XML to JSON conversion, we’ll be using the xml2js module from Node Package Manager (NPM). This package provides a convenient API that does all the heavy lifting for us.

Create an Application

Let us first create a new Node.js application and install all the required dependencies. Switch to the folder where you want to create your project and run the following command in the terminal:

The above command will create a package.json file in the root directory. Next, install the only required dependency — xml2js by executing the following command:

This will create a node_modules directory inside your project that will contain all the dependencies required by the xml2js module.

Convert XML to JSON

To convert an XML string to a JSON object, first create a new JavaScript file named app.js in your project root directory:

If the touch command is not available, just create the file manually. Open the app.js file in your favorite editor and add the following code:

That above example code parses an XML string and converts it into a JavaScript object. To convert the JavaScript object into its JSON representation, we are using the JSON.stringify() method.

To see the output, type the following command to launch the Node.js application:

If everything goes right, you should see the following JSON printed on the console:

The above JSON response looks different than what we usually see when dealing with JSON objects. All the XML string elements are turned into JSON arrays. It is a bit different than what I was expecting i.e. «name»: «John Doe» instead of «name»: [«John Doe»] , but it is XML after all.

The xml2js package provides several options that you can pass as second argument to further customize the output. For example, you can set mergeAttrs: true to merge attributes and child elements as properties of the parent, instead of keying attributes off a child attribute object:

Here is how the JSON output looks now:

As you can see above, id is now merged with the user object and the special $ attribute is removed.

Write JSON to File

To write the output JSON string to a file for further processing, you can use the following JavaScript code:

Take a look at this guide to learn more about writing JSON to a file in Node.js.

Read XML from File

If your XML data is stored in a file, you can read it by using the native fs module and then perform the conversion as shown below:

Take a look at this guide to learn more about reading and writing files in a Node.js application.

JavaScript Promises

Want to use promises instead of the callback? The xml2js module provides a special method called parseStringPromise() that returns a promise as shown below:

Async-Await

The xml2js module also supports async-await syntax as shown in the following example:

Source Code: Download the complete source code from GitHub available under MIT license.

Conclusion

That’s all folks! In this article, we looked at several examples to learn how to complete XML to JSON conversion in a Node.js application by using the xml2js package.

Take a look at xml2js module’s documentation to learn about all the available methods and properties.

✌️ Like this article? Follow me on Twitter and LinkedIn. You can also subscribe to RSS Feed.

Источник

script to translate xml to json

I have 5000 questions in txt file like this:

I want to write a script in Ubuntu to convert all questions like this:

4 Answers 4

I assume your Ubuntu has python installed

save it and chmod to executable you can easily modify to take a file as input instead of just text

EDIT Okey, this is a more complete script:

I made a file with following, which I named xmltest :

So you have a list of quiz inside some other container.

Now, I launch it like this: $ chmod u+x scratch.py , then scratch.py filenamewithxml

This gives me the answer:

in fact, you could get away here even w/o Python programming, just using 2 unix utilities:

1. jtm — that one allows xml json lossless conversion
2. jtc — that one allows manipulating JSONs

Thus, assuming your xml is in file.xml , jtm would convert it to the following json:

and then, applying a series of JSON transformations, you can arrive to a desired result:

Though, due to involved shell scripting ( echo command), it’ll be slower than Python’s — for 5000 questions, I’d expect it would run around a minute. (In the future version of the jtc I plan to allow interpolations even in statically specified JSONs, so that for templating no external shell-scripting would be required, then the operations will be blazing fast)

Источник

How to convert JSON to XML or XML to JSON?

I started to use Json.NET to convert a string in JSON format to object or viceversa. I am not sure in the Json.NET framework, is it possible to convert a string in JSON to XML format and viceversa?

13 Answers 13

Yes. Using the JsonConvert class which contains helper methods for this precise purpose:

Yes, you can do it (I do) but Be aware of some paradoxes when converting, and handle appropriately. You cannot automatically conform to all interface possibilities, and there is limited built-in support in controlling the conversion- many JSON structures and values cannot automatically be converted both ways. Keep in mind I am using the default settings with Newtonsoft JSON library and MS XML library, so your mileage may vary:

XML -> JSON

1. All data becomes string data (for example you will always get «false» not false or «0» not 0) Obviously JavaScript treats these differently in certain cases.
2. Children elements can become nested-object <> OR nested-array [ <><> . ] depending if there is only one or more than one XML child-element. You would consume these two differently in JavaScript, etc. Different examples of XML conforming to the same schema can produce actually different JSON structures this way. You can add the attribute json:Array=’true’ to your element to workaround this in some (but not necessarily all) cases.
3. Your XML must be fairly well-formed, I have noticed it doesn’t need to perfectly conform to W3C standard, but 1. you must have a root element and 2. you cannot start element names with numbers are two of the enforced XML standards I have found when using Newtonsoft and MS libraries.
4. In older versions, Blank elements do not convert to JSON. They are ignored. A blank element does not become «element»:null

A new update changes how null can be handled (Thanks to Jon Story for pointing it out): https://www.newtonsoft.com/json/help/html/T_Newtonsoft_Json_NullValueHandling.htm

JSON -> XML

1. You need a top level object that will convert to a root XML element or the parser will fail.
2. Your object names cannot start with a number, as they cannot be converted to elements (XML is technically even more strict than this) but I can ‘get away’ with breaking some of the other element naming rules.

Please feel free to mention any other issues you have noticed, I have developed my own custom routines for preparing and cleaning the strings as I convert back and forth. Your situation may or may not call for prep/cleanup. As StaxMan mentions, your situation may actually require that you convert between objects. this could entail appropriate interfaces and a bunch of case statements/etc to handle the caveats I mention above.

Источник