An Introduction to Python Sets

Python supports sets which are a collection of unique elements and provide operations for computing set union, intersection and difference.

“The world is a book and those who do not travel read only one page.”
― Augustine of Hippo

1. Introduction

A set is a collection of unique elements. A common use is to eliminate duplicate elements from a list. In addition, it supports set operations like union intersection and difference.

Continue reading “An Introduction to Python Sets”

Getting Started with Scrapy

Learn how to use Python Scrapy to Extract information from Websites.

“When life gives you lemons, chunk it right back.”
― Bill Watterson

1. Introduction

Scrapy is a python-based web crawler which can be used to extract information from websites. It is fast, simple and can navigate pages just like a browser can.

Continue reading “Getting Started with Scrapy”

Listing a Directory With Python

Learn how to list contents of a directory from python. Also, easily find and process files matching conditions from your python program.

“If at first you don’t succeed, destroy all evidence that you tried.”
― Steven Wright

1. Introduction

There are several methods to list a directory in python. In this article we present a few of these along with the caveats for each.

Continue reading “Listing a Directory With Python”

Regular Expressions Examples

Learn how to build regular expression patterns for parsing phone numbers, emails, etc.

. Introduction

Regular Expressions provide a powerful method to handle parsing tasks while handling texts. Whether the problem is a simple search for a pattern, or splitting a string into components, regular expressions are widely used today.

Continue reading “Regular Expressions Examples”

How to Convert Large CSV to JSON

Learn how to use Jackson Streaming to convert a large CSV to JSON.

“We are a way for the cosmos to know itself.”
― Carl Sagan, Cosmos

1. Introduction

Let us today look into converting a large CSV to JSON without running into memory issues. This previous article showed how to parse CSV and output the data to JSON using Jackson. However, since that code loads the entire data into memory, it will run into issues loading large CSV files such as:

Continue reading “How to Convert Large CSV to JSON”

Using Timer Class to Schedule Tasks

Learn how to use the Timer and the TimerTask classes to implement simple task scheduling for your application.

“Humor is reason gone mad.”
― Groucho Marx

1. Introduction

Scheduling tasks to run is a need which sometimes arises in a java program. Maybe you want to run periodic cleanup of some resource. Or check on the status of some job. Or maybe fetch a URL which might not be available the first time.

Continue reading “Using Timer Class to Schedule Tasks”

Pandas Tutorial – Using Matplotlib

Learn how to massage data using pandas DataFrame and plot the result using matplotlib in this beginner tutorial.

“There is only one thing that makes a dream impossible to achieve: the fear of failure.”
― Paulo Coelho, The Alchemist

1. Introduction

Matplotlib is a graphics and charting library for python. Once data is sliced and diced using pandas, you can use matplotlib for visualization. In this starter tutorial, we take you through the steps to do just that.

Continue reading “Pandas Tutorial – Using Matplotlib”

Pandas Tutorial – Grouping Examples

Demonstrates grouping of data in pandas DataFrame and compares with SQL.

“Don’t waste your time with explanations: people only hear what they want to hear.”
― Paulo Coelho

1. Introduction

Let us learn about the “grouping-by” operation in pandas. While similar to the SQL “group by”, the pandas version is much more powerful since you can use user-defined functions at various points including splitting, applying and combining results.

Continue reading “Pandas Tutorial – Grouping Examples”

Pandas Tutorial – SQL-like Data Selection

Did you know that you can perform SQL-like selections with a pandas DataFrame? Learn how!

“Always keep your words soft and sweet, just in case you have to eat them.”
― Andy Rooney

1. Introduction

In this article, we present SQL-like ways of selecting data from a pandas DataFrame. The SELECT clause is very familiar to database programmers for accessing data within an SQL database. The DataFrame provides similar functionality when working with datasets, but is far more powerful since it supports using predicate functions with a simple syntax.

Continue reading “Pandas Tutorial – SQL-like Data Selection”

Pandas Tutorial – Selecting Rows From a DataFrame

Learn the various ways of selecting data from a DataFrame.

“Always and never are two words you should always remember never to use. ”
― Wendell Johnson

1. Introduction

After covering ways of creating a DataFrame and working with it, we now concentrate on extracting data from the DataFrame. You may also be interested in our tutorials on a related data structure – Series; part 1 and part 2.

Continue reading “Pandas Tutorial – Selecting Rows From a DataFrame”