TypeError: unhashable type: 'slice'

import numpy as npimport pandas as pd# Importing the datasetdataset = pd.read_csv('50_Startups.csv')y=dataset.iloc[:, 4]X=dataset.iloc[:, 0:4]# Encoding categorical datafrom sklearn.preprocessing import LabelEncoder, OneHotEncoderlabelencoder_X = LabelEncoder()X[:, 3] = labelencoder_X.fit_transform(X[:, 3])

X is a data frame and can’t be accessed via slice terminology like X[:, 3]. You must access via iloc or X.values. However, the way you constructed X made it a copy… so. I’d use values

# Importing the librariesimport numpy as npimport matplotlib.pyplot as pltimport pandas as pd# Importing the dataset# dataset = pd.read_csv('50_Startups.csv')dataset = pd.DataFrame(np.random.rand(10, 10))y=dataset.iloc[:, 4]X=dataset.iloc[:, 0:4]# Encoding categorical datafrom sklearn.preprocessing import LabelEncoder, OneHotEncoderlabelencoder_X = LabelEncoder()#  I changed this lineX.values[:, 3] = labelencoder_X.fit_transform(X.values[:, 3])

Storing Python dictionaries

Pickle save:

try:    import cPickle as pickleexcept ImportError:  # python 3.x    import picklewith open('data.p', 'wb') as fp:    pickle.dump(data, fp, protocol=pickle.HIGHEST_PROTOCOL)

See the pickle module documentation for additional information regarding the protocol argument.

Pickle load:

with open('data.p', 'rb') as fp:    data = pickle.load(fp)

JSON save:

import jsonwith open('data.json', 'w') as fp:    json.dump(data, fp)

Supply extra arguments like sort_keys or indent to get a pretty result. The argument sort_keyswill sort the keys alphabetically and indent will indent your data structure with indent=N spaces.

json.dump(data, fp, sort_keys=True, indent=4)

JSON load:

with open('data.json', 'r') as fp:    data = json.load(fp)

PorterStemmer vs LancasterStemmer vs SnowballStemmer

PorterStemmer: Most commonly used stemmer without a doubt, also one of the most gentle stemmers. One of the few stemmers that actually has Java support which is a plus, though it is also the most computationally intensive of the algorithms(Granted not by a very significant margin). It is also the oldest stemming algorithm by a large margin.

SnowballStemmer: Nearly universally regarded as an improvement over porter, and for good reason. Porter himself, in fact, admits that it is better than his original algorithm. Slightly faster computation time than the porter, with a fairly large community around it.

LancasterStemmer: Very aggressive stemming algorithm, sometimes to a fault. With Porter and snowball, the stemmed representations are usually fairly intuitive to a reader, not so with Lancaster, as many shorter words will become totally obfuscated. The fastest algorithm here, and will reduce your working set of words hugely, but if you want more distinction, not the tool you would want.

pandas to_csv with no index

Solution:

  1. use index=False while saving your dataframe to CSV file.

    df.to_csv('file_name.csv',index=False)
  2. You can save your dataframe as it is with an index, and while reading you just drop the column unnamed 0 containing your previous index. Simple!
    df.to_csv(' file_name.csv ')
    df_new = pd.read_csv('file_name.csv').drop(['unnamed 0'],axis=1)
  3. Another solution if you want to keep this column as an index.

    pd.read_csv('filename.csv', index_col='Unnamed: 0')

CrownIt Referral Code(rahulp731):

Referral Code: rahulp731

How To Get Signup Bonus

  1. Download Crownit App here.
  2. Crownit Referral Code rahulp731 compulsory.
  3. Crownit Signup Bonus Rs. 100 Crowns.
  4. Crownit Refer and Earn 2 Rush Tickets.
  5. Redeem Crowns Recharge & Vouchers

CrownIT INFO

All users can find out about the Crown it app for nearby offers, offers, gift vouchers of all kinds, mobile recharge, dth recharge, coupons and much more. Now, Crownit’s reference offer has changed. Now users will get a free reload of rs.100 when referring only to 3 friends.

Use the Crownit rahulp731 reference code when registering in the application and get exciting reward points as a union bonus. The Crown it program refers and wins, now it is fashionable because, for each recommendation of Crownit, users will get 1 ticket.

Collect tickets as much as you can and participate in the Crown It Week Rush program, in which users can win smartphones Google Pixel, iPhone7, Google Chrome, domino vouchers, PVR movie vouchers worth Rs.100 to Rs.1000 and many more prizes.

In every 3 successful crown invitations, it grants 100 lifetime credits to the Crown. The users only have to exchange the credits of the crown for mobile recharges, gift vouchers such as BookMyshow, Domino’s, Amazon, KFC, PVR Cinema, etc. and also redeem coupons available in the Crown It application.

[Solved]: ModuleNotFoundError: No module named 'owlready2'

Solution:

 sudo pip3 install owlready2

Stack Trace:

$ sudo pip3 install owlready2

Password:

The directory '/Users/admin/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

The directory '/Users/admin/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

Collecting owlready2

  Downloading https://files.pythonhosted.org/packages/93/5a/5fa13de9cd0f2b15c85eba326e88b61373212afbca6c2682d7b49dd6cbf6/Owlready2-0.13.tar.gz (23.7MB)

    100% |████████████████████████████████| 23.7MB 1.7MB/s

Installing collected packages: owlready2

  Running setup.py install for owlready2 ... done

Successfully installed owlready2-0.13

[Solved]: ModuleNotFoundError: No module named 'xmltodict'

Solution:

$ sudo pip3 install xmltodict

Stack Tracw
Stack Tracw

$ sudo pip3 install xmltodict

The directory '/Users/admin/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

The directory '/Users/admin/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

Collecting xmltodict

  Downloading https://files.pythonhosted.org/packages/28/fd/30d5c1d3ac29ce229f6bdc40bbc20b28f716e8b363140c26eff19122d8a5/xmltodict-0.12.0-py2.py3-none-any.whl

Installing collected packages: xmltodict

Successfully installed xmltodict-0.12.0

[Solved]: ModuleNotFoundError: No module named 'elasticsearch'

Solution:

$ sudo pip install elasticsearch

Stack Trace
Stack Trace

$ sudo pip install elasticsearch

Password:

The directory '/Users/admin/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

The directory '/Users/admin/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

Collecting elasticsearch

  Downloading https://files.pythonhosted.org/packages/b1/f1/89735ebb863767516d55cee2cfdd5e2883ff1db903be3ba1fe15a1725adc/elasticsearch-6.3.1-py2.py3-none-any.whl (119kB)

    100% |████████████████████████████████| 122kB 2.9MB/s

Requirement already satisfied: urllib3>=1.21.1 in /Users/admin/anaconda3/lib/python3.6/site-packages (from elasticsearch) (1.24.1)

Installing collected packages: elasticsearch

Successfully installed elasticsearch-6.3.1

[Solved]: ModuleNotFoundError: No module named 'unidecode'

Solution:

$ sudo pip3 install unidecode

Stack Trace
Stack Trace

$ sudo pip3 install unidecodeThe directory '/Users/admin/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.The directory '/Users/admin/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.Collecting unidecode  Downloading https://files.pythonhosted.org/packages/31/39/53096f9217b057cb049fe872b7fc7ce799a1a89b76cf917d9639e7a558b5/Unidecode-1.0.23-py2.py3-none-any.whl (237kB)    100% |████████████████████████████████| 245kB 6.1MB/s Installing collected packages: unidecodeSuccessfully installed unidecode-1.0.23

[Solved]: ModuleNotFoundError: No module named 'textblob'

Solution:

$ sudo pip3 install textblob

Stack Trace
Stack Trace

$ sudo pip3 install textblobPassword:The directory '/Users/admin/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.The directory '/Users/admin/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.Collecting textblob  Downloading https://files.pythonhosted.org/packages/7c/7d/ad09a26b63d4ad3f9395840c72c95f2fc9fa2b192094ef14e9e720be56f9/textblob-0.15.2-py2.py3-none-any.whl (636kB)    100% |████████████████████████████████| 645kB 8.5MB/s Requirement already satisfied: nltk>=3.1 in /Users/admin/anaconda3/lib/python3.6/site-packages (from textblob) (3.4)Requirement already satisfied: six in /Users/admin/anaconda3/lib/python3.6/site-packages (from nltk>=3.1->textblob) (1.12.0)Requirement already satisfied: singledispatch in /Users/admin/anaconda3/lib/python3.6/site-packages (from nltk>=3.1->textblob) (3.4.0.3)Installing collected packages: textblobSuccessfully installed textblob-0.15.2

[Solved]: ModuleNotFoundError: No module named 'pytypo'

Solution:

$ sudo pip3 install pytypo

Stack Trace
Stack Trace

$ sudo pip3 install pytypo

Password:

The directory '/Users/admin/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

The directory '/Users/admin/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

Collecting pytypo

  Downloading https://files.pythonhosted.org/packages/9f/80/b0578690bcac288cf9af76abecf2fd30978ea75ebe67da817c018c444abb/pytypo-0.3.0.tar.gz (74kB)

    100% |████████████████████████████████| 81kB 2.9MB/s

Installing collected packages: pytypo

  Running setup.py install for pytypo ... done

Successfully installed pytypo-0.3.0

[Solved]: ModuleNotFoundError: No module named 'enchant'

Solution:

sudo pip3 install pyenchant

Stack Trace
Stack Trace

$ sudo pip3 install pyenchant

The directory '/Users/admin/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

The directory '/Users/admin/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.

Collecting pyenchant

  Downloading https://files.pythonhosted.org/packages/4f/c5/5c18df3c5dbf2ce1e6fc8b0fcce1a5dfe7c4ec5ab33b76722fcca9cbfff5/pyenchant-2.0.0-py2.py3.cp27.cp32.cp33.cp34.cp35.cp36.pp27.pp33.pp35-none-macosx_10_6_intel.macosx_10_9_intel.whl (4.3MB)

    100% |████████████████████████████████| 4.3MB 5.0MB/s

Installing collected packages: pyenchant

Successfully installed pyenchant-2.0.0

[Solved]: ModuleNotFoundError: No module named 'ekphrasis'

Solution: 

sudo pip install ekphrasis

Stack Trace
Stack Trace

Collecting ekphrasis

Requirement already satisfied: termcolor in /Users/admin/.local/lib/python3.6/site-packages (from ekphrasis) (1.1.0)

Requirement already satisfied: tqdm in /Users/admin/anaconda3/lib/python3.6/site-packages (from ekphrasis) (4.28.1)

Requirement already satisfied: numpy in /Users/admin/anaconda3/lib/python3.6/site-packages (from ekphrasis) (1.15.4)

Requirement already satisfied: colorama in /Users/admin/anaconda3/lib/python3.6/site-packages (from ekphrasis) (0.4.1)

Requirement already satisfied: matplotlib in /Users/admin/anaconda3/lib/python3.6/site-packages (from ekphrasis) (3.0.2)

Collecting ujson (from ekphrasis)

Requirement already satisfied: nltk in /Users/admin/anaconda3/lib/python3.6/site-packages (from ekphrasis) (3.4)

Collecting ftfy (from ekphrasis)

  Downloading https://files.pythonhosted.org/packages/8f/86/df789c5834f15ae1ca53a8d4c1fc4788676c2e32112f6a786f2625d9c6e6/ftfy-5.5.1-py3-none-any.whl (43kB)

    100% |████████████████████████████████| 51kB 1.5MB/s

Requirement already satisfied: cycler>=0.10 in /Users/admin/anaconda3/lib/python3.6/site-packages (from matplotlib->ekphrasis) (0.10.0)

Requirement already satisfied: kiwisolver>=1.0.1 in /Users/admin/anaconda3/lib/python3.6/site-packages (from matplotlib->ekphrasis) (1.0.1)

Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /Users/admin/anaconda3/lib/python3.6/site-packages (from matplotlib->ekphrasis) (2.3.0)

Requirement already satisfied: python-dateutil>=2.1 in /Users/admin/anaconda3/lib/python3.6/site-packages (from matplotlib->ekphrasis) (2.7.5)

Requirement already satisfied: six in /Users/admin/anaconda3/lib/python3.6/site-packages (from nltk->ekphrasis) (1.12.0)

Requirement already satisfied: singledispatch in /Users/admin/anaconda3/lib/python3.6/site-packages (from nltk->ekphrasis) (3.4.0.3)

Requirement already satisfied: wcwidth in /Users/admin/anaconda3/lib/python3.6/site-packages (from ftfy->ekphrasis) (0.1.7)

Requirement already satisfied: setuptools in /Users/admin/anaconda3/lib/python3.6/site-packages (from kiwisolver>=1.0.1->matplotlib->ekphrasis) (40.7.3)

Installing collected packages: ujson, ftfy, ekphrasis

Successfully installed ekphrasis-0.5.1 ftfy-5.5.1 ujson-1.35

How to Decompress a .bz2 File in Linux/Mac/Unix

bzip2 is mono-threaded, which means it will take a long time to decompress a large file.

To decompress a .bz2 file multithreaded, you can use the free, open-source program lbzip2:

sudo apt-get install lbzip2lbzip2 -d my_file.bz2

-d indicates you wish to decompress the file. It would automatically determine how many threads it will use. To specify the exact number of threads you want to use, use the -n parameter, e.g.:

lbzip2 -d -n 32 my_file.bz2

Read first 1000 lines of a file in python

Python 3

with open("datafile") as myfile:    head = [next(myfile) for x in range(N)]print(head)

Python 2

with open("datafile") as myfile:    head = [next(myfile) for x in xrange(N)]print head

Here’s another way (both Python 2 & 3)

from itertools import islicewith open("datafile") as myfile:    head = list(islice(myfile, N))print head

How to read a large file, line by line, in Python

Two memory efficient ways in ranked order (first is best) –

  1. use of with – supported from python 2.5 and above
  2. use of yield if you really want to have control over how much to read

1. use of with

with is the nice and efficient pythonic way to read large files. advantages – 1) file object is automatically closed after exiting from with execution block. 2) exception handling inside the withblock. 3) memory for loop iterates through the f file object line by line. internally it does buffered IO (to optimized on costly IO operations) and memory management.

with open("x.txt") as f:    for line in f:        do something with data

2. use of yield

Sometimes one might want more fine-grained control over how much to read in each iteration. In that case use iter & yield. Note with this method one explicitly needs close the file at the end.

def readInChunks(fileObj, chunkSize=2048):    """    Lazy function to read a file piece by piece.    Default chunk size: 2kB.    """    while True:        data = fileObj.read(chunkSize)        if not data:            break        yield dataf = open('bigFile')for chuck in readInChunks(f):    do_something(chunk)f.close()


Pitfalls and for the sake of completeness – below methods are not as good or not as elegant for reading large files but please read to get rounded understanding.

In Python, the most common way to read lines from a file is to do the following:

for line in open('myfile','r').readlines():    do_something(line)

When this is done, however, the readlines() function (same applies for read() function) loads the entire file into memory, then iterates over it. A slightly better approach (the first mentioned two methods are the best) for large files is to use the fileinput module, as follows:

import fileinputfor line in fileinput.input(['myfile']):    do_something(line)

the fileinput.input() call reads lines sequentially, but doesn’t keep them in memory after they’ve been read or even simply so this, since file in python is iterable.