YY Every Day

Never say never, because limits like fears, are often just an illusion.

Scraping JavaScript Webpages With Webkit

- - posted in Architecture - tagged by JavaScript, Qt, Spider, python, webkit | Comments

1.Preface

Nowerdays, there are more and more webpages rendered through javascript, tranditional spider such wget, curl is useless.

An alternative solution is webkit, the open source browser engine used most famously in Apple’s Safari browser. Webkit has now been ported to the Qt framework and can be used through its Python bindings.

Surfing Your Keyboard in Bash With Shortcut

- - posted in TechTricks - tagged by bash, command, shortcut | Comments

Nowadays, the more spending time in Bash shell and typing longer commands, the more precious time is in my mind.
So I spent some time to read about “Command Line Editing” in the bash manual and Google the experience old man left.

Well, here’s the new shortcuts I learned:

ps: the marked ones as em are my favourite.

How to Improve the Performance of System

- - posted in TechMinds - tagged by performance, system, webserver | Comments

1,Preface

Happy families are all alike; every unhappy family is unhappy in its own way.
So does System.

Here is just a simple way to find out what’s ‘unhappy’ in your system.

2,Think as time goes

There are lots of ways to thinking and find something.
Thinking as time goes is an easy one.

However, what we are going to talking, in fact, is thinking as data flows.

Data Mining: R, Python, RapidMiner or Matlab

- - posted in Architecture - tagged by R, data mining, matlab, python, scipy | Comments

In work or study, we have more and more problmes, which can be solved by data mining.

Refer to data mining tools, kinds of stuff some out,such as R, spss, mathematica,etc. The most popular packages in the industry are SAS and SPSS, but they are quite expensive, so you might wanna have a free one, and better a open source one.

  • R
  • RapidMiner
  • python
  • Matlab

Workflow Engine in Python

- - posted in Architecture - tagged by python, spiffworkflow, workflow | Comments

1,What’s workflow

A workflow consists of a sequence of connected steps where each step follows without delay or gap and ends just before the subsequent step may begin.

it’s just a complicated finite statemachine

2,Spiff workflow

Spiff Workflow is a library implementing a framework for workflows. It is based on http://www.workflowpatterns.com and implemented in pure Python.

Setting Up Django and Your Web Server With uWSGI and Nginx

- - posted in Architecture - tagged by django, python, uwsgi, webserver | Comments

Recently, there are many modules or system in python.

This post, let’s focus on webserver system

Out of Web Framework for python, the most famous is django. And the most popular httpserver is nginx, apache, lighttpd.

Python is a script language, how does it communicate with httpserver?

The answer is the sgi or cgi,kind of things, which is used to a bridge.

Here is a tuturial which is aimed at the Django user who wants to set up a production web server. It takes you through the steps required to set up Django so that it works nicely with uWSGI and nginx. It covers all three components, providing a complete stack of web application and server software.

Django is a high-level Python Web framework that encourages rapid development and clean, pragmatic design.