Skip to main content

Extracting a single field from a very long json file

Trivial task, but still can save some time to somebody, so I am gladly sharing.
A friend of a mine has an huge json file, and she had to extract all unique value for a field called "title".  The file was too big to be processed from a notepad or an excel.

With those comands, I was able to obtain a clean, unique and sorted list list of all the content.


grep -o -E '"title":"[^"]+",' tmp.json | sort |uniq > output.txt

sed -i 's/"title":"//g' output.txt

sed -i 's/",//g' output.txt


Comments

You can do the same with just one command:
sed -n "s/^.*\"title\":\"\([^\"]*\)\",.*/\1/p" tmp.json | uniq | sort > output.txt

Popular posts from this blog

Building bitcoin/litecoin on mac os x missing EC.h

For my own fun, I was playing around with bitcoin, to check how the parameters of the cryptocurrencies can be modified.

After recent upgrades to my OS, I could not build any longer the bitcoind. Seems like something changed on that side. Now this command

./configure --with-gui=qt5 --enable-debug
Was sistematically producing 
configure: error: OpenSSL ec header missing

EC is the file for elliptic curve cryptography. I had brew correctly configured, and the header files were all correctly present. After some time spent inspecting the issue, on github I was able to find the solution for this problem. I simply add to export the following
export LDFLAGS=-L/usr/local/opt/openssl/libexport CPPFLAGS=-I/usr/local/opt/openssl/include
There is aksi pull request on bitcoin for that, I hope I could save you some time https://github.com/bitcoin/bitcoin/pull/6885/files?diff=split&unchanged=expanded

Hybris setup issue: Unsupported major.minor version 52.0

While I was typing the final "ant all" command for hybris, it miserably failed with the following exception

hybris/bin/platform/resources/ant/antmacros.xml:123: java.lang.UnsupportedClassVersionError: de/hybris/ant/taskdefs/DbDriverValidator : Unsupported major.minor version 52.0 at java.lang.ClassLoader.defineClass1(Native Method)


Even though a "java -version" gave me back version 8, ant is still using $JAVA_HOME variable.
So you have to change it into the script. My suggestion is that you adapt the setantenv.bat script with the JAVA_HOME directly


@echo off
set ANT_OPTS=-Xmx512m -Dfile.encoding=UTF-8
set ANT_HOME=%~dp0apache-ant-1.9.1
set JAVA_HOME=C:/Program Files/Java/jdk1.8.0_101
set PATH=%ANT_HOME%\bin;%PATH%
rem deleting CLASSPATH as a workaround for PLA-8702
set CLASSPATH=
echo Setting ant home to: %ANT_HOME%
ant -version