Csv to json linux
a command tool easily convert json file to csv or xlsx
Latest commit
Git stats
Files
Failed to load latest commit information.
README.rst
jsoncsv : easily convert json to csv or xls[x]
jsoncsv (with mkexcel) is a command tool to convert json file to csv/xlsx file.
It’s simple, and no need user to specify the keys.
cat the raw.json to csv/xls use command line tool
make sure each line of raw json text file is a json object
This is easily and needn’t care the different keys from any two object.
if input file is an json_array, use -A/—array to decode it
another way to convert file step by step
get more options with —help.
see #QuickStart and get more options with —help.
just expand/restore the json, the expand json is one layer json.
mkexcel the expanded json (one layer)
expand json, 展开 json
the output «expand.json» is only one layer json, it can be easy change to csv or xlsx (with mkexcel)
restore the expanded json 重构被展开的json
separator used for combine the keys in the tree
default separator is .
on safe mode, use escape separator to avoid confilct
dump expanded (by jsoncsv) json file to csv or xls file
chose dump type in [‘csv’, ‘xls’] default is ‘csv’
1. dict keys can’t be just array indexes
Источник
Csv to json linux
CSVJSON csv2json() function
Single function csv2json to reliably convert CSV to JSON. Self contained without dependencies. Used to power CSVJSON the online tool found at www.csvjson.com/csv2json. Used by thousands everyday.
Simply call csv2json passing a string to obtain JSON. The string may be CSV (comma separated values), TSV (tab separated values) or semi-colon separated values (typically used in French Excel). It will auto-detect the separator although you may override or force it via the separator option.
Note: In the browser, global namespace CSVJSON is created. It contains the csv2json function.
In both cases, you would get this in the console:
csv2json supports a number of options passed as an optional hash:
- separator : Character which acts as separator. If omitted, will attempt to detect comma , , semi-colon ; or tab \t .
- parseNumbers : If set to true will attempt to convert a value to a number, if possible.
- parseJSON : If set to true will attempt to convert a value to a valid JSON value if possible. Detects numbers, null , false , true , [] and <> .
- transpose : If set to true will pivot the table. The first column becomes the header.
- hash : If set to true will use the first column as a key and return a hash instead of an array of objects.
You can of course test all of these options online on www.csvjson.com/csv2json.
Run the tests in your browser by opening test-browser.html .
Run the tests through node:
JSON2_mod a replacement of JSON with more options to format your JSON. npm package here.
About
Converts CSV to JSON. Powers the most used online tool CSVJSON. Used by thousands everyday.
Источник
Converting CSV to JSON in bash
Trying to convert a CSV file into a JSON
Here is two sample lines :
I would like to get something like :
Here is my script :
I get really strange results :
What is wrong with my code ?
10 Answers 10
The right tool for this job is jq .
emits, given your input:
By the way, a less-buggy version of your original script might look like:
- There’s absolutely no point using cat to pipe into a loop (and good reasons not to); thus, we’re using a redirection ( ) to open the file directly as the loop’s stdin.
- read can be passed a list of destination variables; there’s thus no need to read into an array (or first to read into a string, and then to generate a heresting and to read from that into an array). The _ at the end ensures that extra columns are discarded (by putting them into the dummy variable named _ ) rather than appended to pos .
- «$
- printf is used in preference to echo , as advised in the APPLICATION USAGE section of the specification for echo itself.
- This is still inherently buggy since it’s generating JSON via string concatenation. Don’t use it.
Here’s a python one-liner/script that’ll do the trick:
The accepted answer uses jq to parse the input. This works but jq doesn’t handle escapes i.e. input from a CSV produced from Excel or similar tools is quoted like this:
will result in the incorrect output, as jq will see 4 fields, not 3.
One option is to use tab-separated values instead of comma (as long as your input data doesn’t contain tabs!), along with the accepted answer.
Another option is to combine your tools, and use the best tool for each part: a CSV parser for reading the input and turning it into JSON, and jq for transforming the JSON into the target format.
The python-based csvkit will intelligently parse the CSV, and comes with a tool csvjson which will do a much better job of turning the CSV into JSON. This can then be piped through jq to convert the flat JSON output by csvkit into the target form.
With the data provided by the OP, for the desired output, this as as simple as:
Note that csvjson automatically detects ; as the delimiter, and without a header row in the input, assigns the json keys as a , b , and c .
The same also applies to writing to CSV files — csvkit can read a JSON array or new-line delimited JSON, and intelligently output a CSV via in2csv .
Источник
Csv to json linux
csvtojson module is a comprehensive nodejs csv parser to convert csv to json or column arrays. It can be used as node.js library / command line tool / or in browser. Below are some features:
- Strictly follow CSV definition RF4180
- Work with millions of lines of CSV data
- Provide comprehensive parsing parameters
- Provide out of box CSV parsing tool for Command Line
- Blazing fast — Focus on performance
- Give flexibility to developer with ‘pre-defined’ helpers
- Allow async / streaming parsing
- Provide a csv parser for both Node.JS and browsers
- Easy to use API
Here is a free online csv to json convert service utilizing latest csvtojson module.
csvtojson has released version 2.0.0 .
- To upgrade to v2, please follow upgrading guide
- If you are looking for documentation for v1 , open this page
It is still able to use v1 with csvtojson@2.0.0
From CSV File to JSON Array
From CSV String to CSV Row
Asynchronously process each line from csv url
Convert to CSV lines
To find more detailed usage, please see API section
Command Line Usage
Convert csv file and save result to json file:
Pipe in csv data:
require(‘csvtojson’) returns a constructor function which takes 2 arguments:
- Parser parameters
- Stream options
Both arguments are optional.
For Stream Options please read Stream Option from Node.JS
parserParameters is a JSON object like:
Following parameters are supported:
- output: The format to be converted to. «json» (default) — convert csv to json. «csv» — convert csv to csv row array. «line» — convert csv to csv line string
- delimiter: delimiter used for separating columns. Use «auto» if delimiter is unknown in advance, in this case, delimiter will be auto-detected (by best attempt). Use an array to give a list of potential delimiters e.g. [«,»,»|»,»$»]. default: «,»
- quote: If a column contains delimiter, it is able to use quote character to surround the column content. e.g. «hello, world» won’t be split into two columns while parsing. Set to «off» will ignore all quotes. default: » (double quote)
- trim: Indicate if parser trim off spaces surrounding column content. e.g. » content » will be trimmed to «content». Default: true
- checkType: This parameter turns on and off whether check field type. Default is false. (The default is true if version headName: . e.g.
- alwaysSplitAtEOL: Always interpret each line (as defined by eol like \n ) as a row. This will prevent eol characters from being used within a row (even inside a quoted field). Default is false. Change to true if you are confident no inline line breaks (like line break in a cell which has multi line text).
- nullObject: How to parse if a csv cell contains «null». Default false will keep «null» as string. Change to true if a null object is needed.
- downstreamFormat: Option to set what JSON array format is needed by downstream. «line» is also called ndjson format. This format will write lines of JSON (without square brackets and commas) to downstream. «array» will write complete JSON array string to downstream (suitable for file writable stream etc). Default «line»
- needEmitAll: Parser will build JSON result is .then is called (or await is used). If this is not desired, set this to false. Default is true. All parameters can be used in Command Line tool.
Asynchronous Result Process
Since v2.0.0 , asynchronous processing has been fully supported.
e.g. Process each JSON result asynchronously.
For more details please read:
Converter class defined a series of events.
Источник
Online tool to convert your CSV or TSV formatted data to JSON.
1) Copy/paste or upload your Excel data (CSV or TSV) to convert it to JSON. 2) Set up options: parse numbers, transpose your data, or output an object instead of an array. 3) Convert and copy/paste back to your computer. 4) Save your result for later or for sharing.
Node.js
This function is available as a npm package.
Uploading multiple CSVs?
Embed all the functionality of CSVJSON in any web application with Flatfile, and more. Auto-match columns, validate data fields, and provide an intuitive CSV import experience.
About CSV
- CSV stands for Comma Separated Values. Often used as an interchange data format to represent table records, one per line. CSV is plain text.
- The CSV format is documented in an RFC memo RFC-4180.
- The first line is often the header, or column names. Each subsequent row is a record and should have the same number of fields.
- Fields containing the separator character, line breaks and double-quotes must be enclosed inside double quotes » .
- Other separator are often used like tabs \t or semi-colons ; . TSV or Tab Separated Values is used to store table data in Clipboards. When data is copied from Excel for example, it is stored as TSV in the Clipboard.
- With CSVJSON you can parse values as numbers or JSON. Turn on respective Parse Numbers and Parse JSON switches to convert valid numbers and JSON (null, false, true, [] and <>).
- With CSVJSON you can transpose the csv before conversion. Rows become columns, and columns become rows.
- With CSVJSON you can output a hash (or object) instead of an array. In that case, the hash key will be the first column.
Change Log
- Mar 20, 2019 Bug fix: Detect duplicate column headers and make them unique. GitHub issue #71.
- Feb 2, 2019 Refactored and published npm package csv2json. Fix for GitHub issue #70.
- Jan 26, 2019 Improvement: Removed 64k limit on download button. Also fixed issue #68 ‘Cannot work functional with special letters’.
- Mar 31, 2018 Improvement: Added option to parse JSON values.
- Dec 18, 2017 Improvement: Added option to minify or compact JSON. GitHub issue #21
- Oct 7, 2016 Improvement: Added option to parse number values or not to retain original number formatting. GitHub issue #13
- Jul 09, 2016 Fixed bug : If no text is present in a csv field, it was assigned 0 (zero) by default.
- Jun 20, 2016 Bug fix: strings containing quotes and commas were prematurely cut.
- Dec 30, 2015 Bug fix: drop quotes on keys of nested objects.
- Nov 26, 2015 Improvement: Added options to transpose and output object instead of array.
- Jan 30, 2014 Bug fix: Pasting Excel data into Textarea would cause an upload.
- Jan 12, 2014 Initial release.
Feedback
Code available on GitHub. Report bugs or ask for improvements through GitHub issues.
Источник
