PiPlanter 2 | Solving Broken Pipe Errors [Errno 32] in Tweepy

If I haven’t mentioned it already, https://twitter.com/piplanter_bot IS the new twitter account for PiPlanter. Like last time, I’m using the tweepy library for python to handle all things twitter for the project. What I’m NOT using this time is Flickr. From a design point of view, it wasn’t worth it. It was too complicated and had too many things that could go wrong for me to continue using it. Twitter is more than capable of hosting images, and tweepy has a very simple method of passing these images to twitter. Recently I moved the whole setup indoors and mounted it all onto a shelf seen here and it came with a set of strange problems.

Long story short, what I think happened was that since I moved them to a different location, the complexity of the images increased, causing an increase in the size of the images themselves. A broken pipe error implies that the entirety of the package sent to twitter wasn’t sent, causing the tweet not to go through. I first started to suspect this problem after seeing this:

 

The graphs were going through just fine, but images were seeming to have a hard time. You can’t tell from this photo, but those tweets are hours apart as opposed to the 20 minutes they are supposed to be. Once I started having this problem, I bit the bullet and integrated logging into my project which produced this log:

Hours and hours of failed tweets due to “[Errno 32] Broken pipe”. I tried a lot of things, I figured out that it was the size of the images after seeing this:

Photos that were simple in nature had no problem being sent. After scaling the image size down, I’ve had absolutely no problem sending tweets.


If you are tweeting images with tweepy in python and getting intermediate Broken pipe errors, decrease the size of your image.
Thanks for reading.

Hey! This post was written a long time ago, but I'm leaving it up on the off-chance it may help someone. Proceed with caution. It may not be a good idea to blindly integrate this code or work into your project, but instead use it as a starting point.

PiPlanter 2 | Progress Update

I’m almost done with a very stable version of the Python code running the PiPlanter. There are many specific differences between this version of the python code and the version I wrote and implemented last summer, but the main one is that I tried to write functions for pretty much every task I wanted to do, and made each routine much more modular instead of one long line after line block to do each day. This took significantly longer to do (thus the lack of updates, sorry) but is much more expandable going forward. Below is the new version of the code, but by no means am I an expert programmer. The following code seems to work very well for what I want it to do.

Note the distinct lack of comments. I will put out a much more polished version of the code when it’s done. Before I move onto things like a web UI etc, I would like to do a few more things with this standalone version. The above version renders videos into time lapses, I would like to be able to upload those videos somewhere, hopefully youtube. I would also like to be able to email the log file to the user daily, which should be easier than uploading videos to youtube.

The script that renders the MySQL data into a graph is the following, it on the other hand has not changed much at all since last year and is still the best method to render graphs like I want to:

Here are some photos of the current setup, it hasn’t changed much since last time:

Thank you very much for reading.

Hey! This post was written a long time ago, but I'm leaving it up on the off-chance it may help someone. Proceed with caution. It may not be a good idea to blindly integrate this code or work into your project, but instead use it as a starting point.

Server Upgrade | Stress Testing

So I wrote a program that makes really big numbers in python in an attempt to break a VM. Here’s the code:

I left it on for a few days and ended up with this:

Someone try and beat 26 compounds!

Hey! This post was written a long time ago, but I'm leaving it up on the off-chance it may help someone. Proceed with caution. It may not be a good idea to blindly integrate this code or work into your project, but instead use it as a starting point.

DSFU – Adding Email Functionality, Better User Experience, Stable Set Adding

Big post for this project, here’s a video:

This version of the code implements a few really cool features.

First things first I added 10 LEDs that display the percent uploaded of the batch. For example if 13 / 100 photos have been uploaded, the first LED will light up. If 56 / 100 the first 5 LEDs will light up. Eventually the 10 junk LEDs will be replaced with a bar graph which will be mounted externally on the front panel of the enclosure.

I am using every single available output on my Pi now, but I was able to get away with adding 1 more LED that I should be able to use by using a transistor array explained here:

On the code side of things, I updated the way photos are added to the set. It uses the same principal as described in the previous post (using APscheduler to do the adding on an interval). All of these changes can be seen below, it’s still very poorly commented however.

Thanks for reading!

Hey! This post was written a long time ago, but I'm leaving it up on the off-chance it may help someone. Proceed with caution. It may not be a good idea to blindly integrate this code or work into your project, but instead use it as a starting point.

PiPlanter | Big Overhaul Update

Okay! So I leave for college in less than 30 days, but I’d like to make sure my tomatoes to continue to grow once I leave so I’ve taken some steps to make sure that my departure goes smoothly.

Here’s a video of my revised setup:

There are a few key differences between this setup and my previous one:

The main one is that the watering system has been 100% re-vamped. The water distribution happens via a hose with holes in it instead of using the tray at the bottom of the plant grid in the previous video.

It also takes, uploads and tweets a picture of itself using a raspberry pi camera module.

It also creates a new mysql table every two weeks, and in turn, renders a new kind of graph. The renderscript.php file receives an argument from the python script which is the table code.

Here’s the python script:

Here’s the .php script:

Thank you for reading!

Hey! This post was written a long time ago, but I'm leaving it up on the off-chance it may help someone. Proceed with caution. It may not be a good idea to blindly integrate this code or work into your project, but instead use it as a starting point.

PiPlanter – A Plant Growth Automator

New Version The Post Below Is Out Of Date Click Here For The New Version


This post is many months in the making and I am very proud of the thing’s I’ve done here, and very thankful to all of those (specifically at www.reddit.com/r/raspberry_pi) who have helped me along my way to getting this project up and running.

This page contains every single post related to this project, please feel free to go back and look at my progression and pick up tips along the way if you want to try something like this.

Let’s get this going, here’s an overview video:

There are 8 parts to this system and, you guessed it, I’ll be going in-depth about every single one!

Sensor Network

So at it’s core, the PiPlanter is a Sensor Network & Pump System. Here’s a video explaining the sensor array:

This project uses a TMP35-37 sensor to get a pretty precise temperature reading of the room. Later down in this post you can find out the algorithm to determine the temperature in Fahrenheit. It also uses a basic LDR to get the relative ambient light level in the room. Along with those two sensors, there are 4 relative humidity sensors of my own design, here’s a picture of them as seen in this post:

9allassemb

They’re hooked up to the ADC (mentioned later) in the same way that the LDR is, with a voltage dividing resistor, and then fed directly into ADC. The principal behind this sensor is that when you insert it into soil, the water in that soil connected the two probes, causing a voltage to flow across them. So if there is more water in the soil, more electrons will flow across them, and the analog value will be higher. It’s very basic, but it works. I’ve done several long term tests, and over time, as the soil becomes dryer, the value gets lower, indicating relative dryness. Here is a picture of the four probes in the soil, with the plants.

The TMP sensor’s output is plugged directly into the ADC and the LDR is very basically connected to the ADC as well, this is essentially how how the whole thing is setup on the breadboard:

Capture

Pump System

The pump system is pretty dead simple. Essentially it is a PowerSwitch Tail II switching the mains to a 9v DC power supply. The 9v power supply is connected directly to a 12v DC submersible pump. Instead of using a motor driver chip, which requires 3 pins to do, and the chip would get hot and whatnot, I’ve decided to go with this method.

The pump is not self priming. This means it cannot make the transition from pumping air to pumping water. I wrestled with this problem for a long time, and came up with what I think is an elegant solution. I submerged the pump directly into the water, which means the pump will never fill with air, and will always pump water when activated. Here’s a video explaining the pump system:

Raspberry Pi ADC

The next system is the ADC connected to the Raspberry Pi. It is an 8 bit, 8 port analog to digital converter that can easily run on 3.3v so it’s perfect for the pi. Here is the chip, and you set it up as follows (I took this from an earlier post I wrote)

Now we need to set up the specific libraries for python the first of which being spidev, the spi tool for the raspberry pi which we can grab from git using the following commands:

You also need to (copied from http://scruss.com/blog/2013/01/19/the-quite-rubbish-clock/):

As root, edit the kernel module blacklist file:

Comment out the spi-bcm2708 line so it looks like this:

Save the file so that the module will load on future reboots. To enable the module now, enter:

To read from the ADC, add the following to your python code. The full code will be listed later:

So just use “readadc(n)” to get a value.

Python Code

I’ve made a real effort this time to comment my code well, so I’m not going to do a line by line breakdown like I often do, but I will clearly state the installs and setup things as follows. I’m assuming you have python-dev installed.

Download and install: APScheduler, this is a very straight forward install

Download and install: tweepy, you will need to go through the API setup process.

Download and install: flickrapi, you will need to go through the API setup process.

Here’s the source code for the python component of this project:

There you go! Essentially, every hour, the raspberry pi samples data from 4 humidity probes, an LDR and a tmp sensor. Once the sampling is complete, it dumps the data into a mysql database. From there the data is rendered into a graph using pChart in the form of a .png image. From there, that .png files is uploaded to flickr using this api. Once the file is uploaded, it returns it’s photo ID to the python script. From there, a tweet is built containing the brightness at the time of the tweet, the temperature at the time of the tweet, and the average moisture of the plants. It also uses the photo ID from flickr obtained earlier to build a URL leading to that image on flickr which it tweets as well. The final part of the tweet is a url that leads to this post! (taken from)

MySQL Database

The database is extremely simple, after installing MySQL set it up and create table that follows this syntax:

Pretty basic stuff, the table is just where the python script dumps the data every hour.

PChart Graph

The software driving the graphing part of the project is a bit of php graphing software called pchart. It allows me to graph mysql values from a table in a variety of ways. It is very important, and the code for the php script is as follows:

As you may be able to guess, upon the calling of this script, the program looks for a table called “piplanter_table_17” and does a bunch of stuff as commented to produce a graph. This is what a sample graph looks like:

Wed Jun 26 19:39:17 2013

This is data taken over 6 days, and it’s a lot to look at, but it’s good stuff.

Twitter & Flickr Integration

As you hopefully derived from the python code, this project uses Twitter to send data to me. Instead of using an email server or sending sms messages, I decided on twitter because of a few reasons. I use the service constantly, so I won’t ever miss a tweet. The API seemed really easy to use (and it was!) and allowed more than one person to acess the data at any one time. I decided to use flickr as my image hosting service for a lot of the same reasons, but the main one was their 1TB storage per person. You’ve already seen a sample flickr upload, so here’s a sample tweet:

That’s essentially it! Thank you for reading, and please ask questions.

Hey! This post was written a long time ago, but I'm leaving it up on the off-chance it may help someone. Proceed with caution. It may not be a good idea to blindly integrate this code or work into your project, but instead use it as a starting point.

PiPlanter | Bringing most of it together

Last night I finished the majority of the software for this project. Here’s a video of me going over what happened and what the program does in simpler terms:

Essentially, every hour, the raspberry pi samples data from 4 humidity probes, an LDR and a tmp sensor. Once the sampling is complete, it dumps the data into a mysql database. From there the data is rendered into a graph using pChart in the form of a .png image. From there, that .png files is uploaded to flickr using this api. Once the file is uploaded, it returns it’s photo ID to the python script. From there, a tweet is built containing the brightness at the time of the tweet, the temperature at the time of the tweet, and the average moisture of the plants. It also uses the photo ID from flickr obtained earlier to build a URL leading to that image on flickr which it tweets as well. The final part of the tweet is a url that leads to this post!

That was a lot of explanation, but this program does quite a bit. The source comes in two parts, here’s the python script that handles the brunt of the processing. You will need a bunch of libraries to run this, you could pick through past posts of mine to find what those are, but when I do a final post for this project I will include all of those.

Here’s the .php script that renders the graph from the mysql data. It is called by the python script.

Thanks for reading!

Hey! This post was written a long time ago, but I'm leaving it up on the off-chance it may help someone. Proceed with caution. It may not be a good idea to blindly integrate this code or work into your project, but instead use it as a starting point.

Integrating twitter to the Raspberry Pi

Here’s a video of the system working:

I wanted to create a way to push data from my Raspberry Pi monitoring plant growth to myself. Instead of creating an email server and sending emails, or setting up an sms client, I decided to install tweepy and use twitter and python to send me the data.

First thing’s first, I had to create a dummy account (@eso_rpi) and sign up for the Twitter Dev Program, which is a free way to access the API. You will need to generate a set of consumer keys and access tokens for your app. The process is pretty simple, and the tweepy example is pretty straight forward. If you run into trouble you could easily google it as the process is pretty well documented.

Here’s my code, you will need to download and install tweepy and apscheduler for this to work:

The wiring diagram is the same as it is here, except there is an ldr connected to port 1:

There you go!

Hey! This post was written a long time ago, but I'm leaving it up on the off-chance it may help someone. Proceed with caution. It may not be a good idea to blindly integrate this code or work into your project, but instead use it as a starting point.

PiPlanter | Second round of data collection

So as I said in one of my previous posts, I am going to be collecting a lot of data over the next few weeks while the tomato plants grow. I will be doing this to determine when soil is “dry” and how temperature and light effect that process. For the last week I have been collecting data in the configuration seen in my last post and here is the graph it produced you can click to see the full image:

This graph proves a few things. The first thing is that the relative moisture sensor works. As one can intuitively understand, if you don’t add more water into the system, nature will remove water via evaporation. The overall trend of the blue line (the rel mst sensor) is downward, backing up this point.

The problem with this setup was that I was spitting the voltage across the two probes constantly, which along with the water caused the nails to rapidly oxidize, which is something I would like to avoid in the long term. This also may have seriously corrupted the data so besides general trends, this whole set is unusable.

This isn’t necessarily a bad thing though, as I wanted to conduct a second trial