Posts

Google App Engine VirtualEnv Tool that is not virtualenv for Python

When I build app engine projects and I want to include python package, what I do is download the source and symlink/include it in the project. I have tried the virtualenv approach but it didn't feel clean and I couldn't find anything that will suit my needs in a simple way. So I created this new tool called gaenv which will automate installation/linking of your installed packages to your gae project, it doesn't really need to be a gae project since what it does is just create a folder of symlinks following your requirements.txt so this can be useful to any container base deployment packages that follows symbolic links. Installation & usage: # Note that if you install on a system with multiple python, # you need to call the correct binary on which it looks up packages $ sudo pip install gaenv $ cd to/your/gae-project # Create your requirements.txt & run this to install it $ pip install -r requirements.txt $ gaenv That's it, this ...

Windows Process Monitor Script in Python

I have a windows desktop that is always tunneled to some server for reason that is not important. I use kitty.exe an improved putty with auto re-login and more, but only needed that feature that's why I'm using that. But for some reason it stops responding if a lot of traffic has passed to the tunnel and it has now happened too frequent that I just had to auto restart it, there might be softwares that does this already but here is what I did using python. This should be reusable with any process just change the variable names you want to monitor. import os import subprocess import time __author__ = 'faisal' # Change this if you want to monitor a different process process_name = 'kitty.exe' # Separate your parameters start_command = ['C:\Applications\Putty\kitty.exe', '-load', 'MyTunnelSession'] # Sleep time in seconds before checking again sleep_time = 60 while True: # Filter a list of windows process that has stopped respondi...

Backup/Sync Your Photos to Flickr Script

IF YOU JUST WANT TO USE IT PLEASE SKIP TO INSTALL PART If you haven't heard, flickr has made their free storage 1TB for your full size photos. That's a lot of storage for your photos. I have a collection of photos since year 2000, and I have only around 81GB of photos stored to my hard drive and time machine and another backup drive, but those can all fail but I have been lucky not to lose photos from hard drive failures. I was looking for a sync apps to flickr that I can leave alone and I couldn't find one. So here is a quick python script that I created. Here is how it works for backing it up: Let's say you have folder /Volumes/Storage/Pictures/ 2001   / 2001-01-01     / File1.jpg     / File2.jpg 2002 2003 When you run flickrsmartsync it will require you to login give it access then it will sync it automatically, stopping and starting it will just resume it as long as it is in the same structure. So if you sync the same structure under the roo...

Simple Mapper Class for NDB on App Engine

This class is based on the db mapper found in remote_api article. But using ndb, the purpose of this is if you want to iterate through a lot of entities but not enough time to do it on request time. So this library helps you create a map of your entities of given kind. You should use this in cases like, deleting users that requested for deletion or updating counters for specific filters. Here is the NDB version of the Mapper. I have added a bit of improvement that I have used for in the past. -Edit- I have removed the memcache ability to stop duplicates. It should just now be handled with the task scope, like taskname or different filters per task (can be done with different initial data and overriding the get_query method and use it as some filter). import logging from google.appengine.ext import deferred, ndb from google.appengine.runtime import DeadlineExceededError class Mapper(object): def __init__(self, use_cache=False): ndb.get_context().set_cache_policy...

Summary of my Android Apps

After checking the archive list of my blog I didn't see me sharing my android apps on here. So here it is, I currently have 4 active apps mostly created for myself that I published on google play. AppLauncher+ This app automatically organize your apps base on google play categories. Reason I built is cause I flash my firmware a lot at one time that reorganizing folders of my apps just takes too much time. I couldn't find one that is simple enough that would just work and I won't ever touch it again. It has now evolve to have features like: Manual Categorization (had to do it cause of too much demand) Floating launcher (for paid, you can open a folder/assign commands on what it does) Create Shortcut & Folder view on those shortcut (also paid only) Free version basically just gets an organized list with ads! :( Live Battery (Status Bar) This is a live wallpaper, you can select a static wallpaper then then it will have your borders as status bar. I did this...

JSONRPC Server & Client For Python on Google App Engine

Now that google cloud endpoints is around the corner it will and probably should be a standard way of creating web services for any types of clients for mobile, desktop or even your ajax requests. It's still experimental as the time of this writing and I will not really talk about how to use it since their documentation has some good example on it already. I will be sharing on how and what I've used to create my own web services for android clients I have created and for ajax calls. I have created my own jsonrpc client/server class for python. My own full implementation of jsonrpc standards . I have included this on my app-engine-starter code with some sample if you run it and click the JSONRPC Demo dropdown. Feel free to use it. It is still a nice simple library to use creating web services. I will give a quick sample code here on how it's used: import logging from google.appengine.ext import webapp, ndb import jsonrpc class Calculator(): def add(self, a,...

NDB Caching Queries Tips & Best Practice - Google App Engine

Update: Since keys only queries are now free, I would prefer to just cache the queries with only resulting to keys_only=True then retrieving the cached values of it with ndb.get_multi(keys). If you are creating a heavy read app engine app, that has a lot of listing/query entities it's a good idea to cache those queries so you don't get charged for reads. But you want it to also be up to date and not have to worry about invalidations. Here is some of the things I've done for caching queries. This can't be applied to all but should work on most and can be implemented on same manner with more complex queries. The idea is to have an updated field on the fields you are filtering from so you can use that as your cache key. Here is a sample code that that shows how to display user post with cached queries. from google.appengine.ext import ndb class User(ndb.Model): created = ndb.DateTimeProperty(auto_now_add=True, indexed=False) updated = ndb.DateTimeProper...