August 21, 2008
buzz
When I was going up the stairs
I met a man who wasn't there
He wasn't there again today
I wish I wish he'd go away
August 20, 2008
心若疲惫,连笑都颓废.
從海南回來二天感到疲憊不堪,似乎怎麼休息都補不回來.
工作愈來愈得心應手,這得益於理念上的認知,當知到一些最基本的知識後,懂得舉一反三.人似乎就開懷起來了.
人生不過是饑來餐,渴來飲,倦來眠.再簡單不過.
沙翁有句話說得好,幸福的家庭都是類似的,不幸的家庭卻各有各的不幸.慢慢去體會好了.
上傳唯一一張用相機在海南拍攝的圖片,攝於天涯海角.
Back from the travel
Nothing fun, even the diving is not cool.
August 11, 2008
what does it look like?
August 4, 2008
ighttpd的performance文档
========================
Performance Improvements
========================
------------
Module: core
------------
:Author: Jan Kneschke
:Date: $Date: 2005-03-28T08:30:05.699628Z $
:Revision: $Revision: 227 $
:abstract:
handling performance issues in lighttpd
.. meta::
:keywords: lighttpd, performance
.. contents:: Table of Contents
Description
===========
Performance Issues
------------------
lighttpd is optimized into varying directions. The most important direction is
performance. The operation system has two major facilities to help lighttpd
a deliver its best performance.
HTTP Keep-Alive
---------------
Disabling keep-alive might help your server if you suffer from a large
number of open file descriptors.
The defaults for the server are: ::
server.max-keep-alive-requests = 128
server.max-keep-alive-idle = 30
server.max-read-idle = 60
server.max-write-idle = 360
handling 128 keep-alive requests in a row on a single connection, waiting 30 seconds
before an unused keep-alive connection gets dropped by lighttpd.
If you handle several connections at once under a high load (let's assume 500 connections
in parallel for 24h) you might run into the out-of-fd problem described below. ::
server.max-keep-alive-requests = 4
server.max-keep-alive-idle = 4
would release the connections earlier and would free file descriptors without a
detrimental performance loss.
Disabling keep-alive completely is the last resort if you are still short on file descriptors: ::
server.max-keep-alive-requests = 0
Event Handlers
--------------
The first one is the Event Handler which takes care of notifying the server
that one of the connections is ready to send or receive. As you can see,
every OS has at least the select() call which has some limitations.
============ ========== ===============
OS Method Config Value
============ ========== ===============
all select select
Unix poll poll
Linux 2.4+ rt-signals linux-rtsig
Linux 2.6+ epoll linux-sysepoll
Solaris /dev/poll solaris-devpoll
FreeBSD, ... kqueue freebsd-kqueue
============ ========== ===============
For more information on this topic take a look at http://www.kegel.com/c10k.html
Configuration
`````````````
The event handler can be set by specifying the 'Config Value' from above
in the ``server.event-handler`` variable
e.g.: ::
server.event-handler = "linux-sysepoll"
Network Handlers
----------------
The basic network interface for all platforms at the syscalls read() and
write(). Every modern OS provides its own syscall to help network servers
transfer files as fast as possible.
If you want to send out a file from the webserver, it doesn't make any sense
to copy the file into the webserver just to write() it back into a socket
in the next step.
sendfile() minimizes the work in the application and pushes a file directly
into the network card (ideally).
lighttpd supports all major platform-specific calls:
========== ==========
OS Method
========== ==========
all write
Unix writev
Linux 2.4+ sendfile
Linux 2.6+ sendfile64
Solaris sendfilev
FreeBSD sendfile
========== ==========
The best backend is selected at compile time. In case you want to use
another backend set: ::
server.network-backend = "writev"
You can find more information about network backend in:
http://blog.lighttpd.net/articles/2005/11/11/optimizing-lighty-for-high-concurrent-large-file-downloads
Max Connections
---------------
As lighttpd is a single-threaded server, its main resource limit is the
number of file descriptors, which is set to 1024 by default (on most systems).
If you are running a high-traffic site you might want to increase this limit
by setting ::
server.max-fds = 2048
This only works if lighttpd is started as root.
Out-of-fd condition
-------------------
Since file descriptors are used for TCP/IP sockets, files and directories,
a simple request for a PHP page might result in using 3 file descriptors:
1. the TCP/IP socket to the client
2. the TCP/IP and Unix domain socket to the FastCGI process
3. the filehandle to the file in the document root to check if it exists
If lighttpd runs out of file descriptors, it will stop accepting new
connections for awhile to use the existing file descriptors to handle the
currently-running requests.
If more than 90% of the file descriptors are used then the handling of new
connections is disabled. If it drops below 80% again new connections will
be accepted again.
Under some circumstances you will see ::
... accept() failed: Too many open files
in the error log. This tells you there were too many new requests at once
and lighttpd could not disable the incoming connections soon enough. The
connection was dropped and the client received an error message like 'connection
failed'. This is very rare and might only occur in test setups.
Increasing the ``server.max-fds`` limit will reduce the probability of this
problem.
stat() cache
============
A stat(2) can be expensive; caching it saves time and context switches.
Instead of using stat() every time to check for the existence of a file
you can stat() it once and monitor the directory the file is in for
modifications. As long as the directory doesn't change, the files in it
must all still be the same.
With the help of FAM or gamin you can use kernel events to assure that
your stat cache is up to date. ::
server.stat-cache-engine = "fam" # either fam, simple or disabled
See http://oss.sgi.com/projects/fam/faq.html for information about FAM.
See http://www.gnome.org/~veillard/gamin/overview.html for information about gamin.
Platform-Specific Notes
=======================
Linux
-----
For Linux 2.4.x you should think about compiling lighttpd with the option
``--disable-lfs`` to disable the support for files larger than 2GB. lighttpd will
fall back to the ``writev() + mmap()`` network calls which is ok, but not as
fast as possible but support files larger than 2GB.
Disabling the TCP options reduces the overhead of each TCP packet and might
help to get the last few percent of performance out of the server. Be aware that
disabling these options most likely decreases performance for high-latency and lossy
links.
- net.ipv4.tcp_sack = 0
- net.ipv4.tcp_timestamps = 0
Increasing the TCP send and receive buffers will increase the performance a
lot if (and only if) you have a lot of large files to send.
- net.ipv4.tcp_wmem = 4096 65536 524288
- net.core.wmem_max = 1048576
If you have a lot of large file uploads, increasing the receive buffers will help.
- net.ipv4.tcp_rmem = 4096 87380 524288
- net.core.rmem_max = 1048576
Keep in mind that every TCP connection uses the configured amount of memory for socket
buffers. If you've got many connections this can quickly drain the available memory.
See http://www.acc.umu.se/~maswan/linux-netperf.txt for more information on these parameters.
FreeBSD
-------
On FreeBSD you might gain some performance by enabling accept filters. Just
compile your kernel with: ::
options ACCEPT_FILTER_HTTP
For more ideas about tuning FreeBSD read: tuning(7)
Reducing the recvspace should always be ok if the server only handles HTTP
requests without large uploads. Increasing the sendspace would reduce the
system load if you have a lot of large files to be sent, but keep in mind that
you have to provide the memory in the kernel for each connection. 1024 * 64KB
would mean 64MB of kernel RAM. Keep this in mind.
- net.inet.tcp.recvspace = 4096
August 3, 2008
The revival of blogspot
Accidentally, I realized that blogspot happens to be accessible when searching google. I can never understand the way the dpi's policy on these things. But whatever, who cares? Blogspot will be gone in days anyway in my opinion.
Cross Building FreeBSD
Originated from here.
I've been getting more and more questions about FreeBSD embedded systems: What is supported? How can I build for them? What can be done to improve the cross development environment? How can I help? and so on. These are all very good questions, so I thought I'd start a series of articles on the FreeBSD build system and embedding FreeBSD into products.
For this article, I'll be providing background on the FreeBSD build system as it relates to embedded systems. For an emebedded system, one usually has little more than a microcontroller with an MMU to power the entire system. While these systems work well in their target niche, they don't work so well as a general purpose platform. They lack the memory and I/O bandwidth necessary to compile applications that run on them. Server machines (or even workstations) do a much better job at these tasks, but are for a different computer architecture. One is left with the choice of compiling natively on a slow machine, or cross building on a fast machine. I'll discuss some basic aspects of cross compilation in this article.
I'll assume that you are familiar with FreeBSD's build system. If not, there are many resources available on the web. One of the better ones is the FreeBSD handbook. Future articles will build on this base, hopefully with sufficient links.
Each FreeBSD platform has a MACHINE and a MACHINE_ARCH that uniquely define it. The MACHINE_ARCH is the CPU family that executes the object code. MACHINE_ARCH is something like "i386" or "sparc64." Some CPU families come in different flavors that are mutually incompatible (such as MIPS processors using different byte ordering). These families would get a different MACHINE_ARCH for each flavor (so you'd have mipseb and mipsel, for example, to distinguish the two different flavors of MIPS CPUs). If you are familar with the gnu compilers and such, this field roughly corresponds to "CPU" field of their "CPU-MANUFACTURER-OS" identifier. I say "corresponds" because FreeBSD uses "amd64" as the MACHINE_ARCH for AMD's extensions to the x86 architecture (the official name of the architecture), while gcc uses the older x86_64 designation.
MACHINE defines the way the machine is put together. Often times machines are build with the same CPU, but with differing glue chips and support software. In cases like this, if the machines are very similar, FreeBSD will support that with one MACHINE type. In other cases, the differences are large enough to warrant an entirely separate MACHINE for that machine. What differentiates one MACHINE type from another with the same MACHINE_ARCH are things like BIOS interface, boot procedure, disk partitioning, and expansion bus technologies. For example, the FreeBSD/i386 and FreeBSD/pc98 platforms both execute on i386 compatible CPUs, but they have completely different bus topologies, boot procedures, BIOS interfaces, and disk layout.
When one builds FreeBSD on one platform to execute on another platform, that is called cross compiling. Cross compilation support is integrated into the normal FreeBSD build system. It is activated by defining TARGET and TARGET_ARCH of the machine you are targeting and using the normal make targets. For example:
make TARGET=arm TARGET_ARCH=arm buildworld
will build the entire userland portion of the system for an ARM machine. Similarly, the following:
make TARGET=arm TARGET_ARCH=arm installworld DESTDIR=/foo
will install these components into the directory tree /foo. One can even build and install a kernel for the target platform:
make TARGET=arm TARGET_ARCH=arm KERNCONF=BONGO kernel DESTDIR=/foo
After cross compilation support was added to the tree, the FreeBSD project added two additional make targets which allow use of the build tools outside of the build tree. The first is "buildenv." It forks a shell with the same build environment used to build FreeBSD. This target is for interactive development. Much of the Atmel AT91RM9200 port was done using this target, for example. However, there's no ability to specify a command to run in this sub-shell, nor does it lend itself well to automated building systems. The make target "buildenvvar" was created to try to fill the gap in the latter problem. This target prints the build environment variables used by the build system. We used it at my day joy to help automate building our products on an x86 processor for an embedded ARM design we're producing. This works well for many applications, but not for all. We have found that we can make it work for simple applications, but more complicated one, with more elaberate build systems, require something more sophisticated. In a future article, I'll describe how we solved that problem.
Many times, cross building can be useful for "compile testing" changes to the tree. There's a make target called "universe" that builds the entire tree for each of the Tier 1 and 2 architectures, along with all the kernels that it can find. In many cases, these "compile tests" of the system find problems that would otherwise be found by others, thus saving much embarassment. The drawback, however, to a "universe" build is that it takes several hours on a very fast machine.
I hope that you've found the above information useful, and that it answers your basic questions in this area. Feel free to leave feedback with additional questions or requests for clarification and I'll see what I can do about addressing them in the feedback section, or in future articles.
Recommended NICs
These adapters are supported by the em(4) drivers:
FreeBSD IP Stack Tuning Simplified!
Author: Diesel@bsdvault.net
The Following is the basic structure I use when Tuning my FreeBSD tcp stack to optimize performance. Most of what follows is my deduction from UNIX IP Stack Tuning Guide v2.7 written by Rob Thomas. I have found this to be very effective way of increasing performance on my box. lets begin!
First lets understand that the following will be managed with sysctl commands being run from /etc/sysctl.conf at startup. This is a very easy way to manage your kernel and how you want your stack to be modified. If anyone has suggestions on how to improve the following, please email me at diesel@bsdvault.net. Furthermore I know several of you may not know of or understand sysctl command. You can find a tutorial on the command itself here as well. You should do fine without really understand sysctl except to know that it allows you to dynamically load kernel modules.
UNIX IP Stack Tuning Guide v2.7
By Rob Thomas, robt@cymru.com, 03 DEC 2000
Introduction
The purpose of this document is to strengthen the UNIX IP stack against a variety
of attack types prevalent on the Internet today. This document details the
settings recommended for UNIX servers designed to provide network intensive
services such as HTTP or routing (firewall services). This document covers
the following UNIX variants:
