Thomas Gericke

Hi there!

Because some of you asked, how I realized the grabbing and thumbnailing of whole websites (here’s an example and I wrote about that in this post), this is a brief HOWTO.

Imagine, you have a Linux system without graphical support. How do you display complex graphical content and make a screenshot? Here it comes: grabbing websites on a Linux system is quite simple.

Prerequisites:

1. a Linux operating system (Debian is fine)
2. khtml2png (I used khtml2png_2.7.6_i386.deb from here)
3. a running X server (Xvfb does it for me)
4. kdelibs4c2a
5. libkonq4

This is it!

The trick now is: on a system working as a server, you usually don’t want to have a running X server. So, I just installed Xvfb, which is a “Virtual Framebuffer ‘fake’ X server”. It is running in the background and khtml2png uses its display.

First, install Xvfb and several libs:

apt-get install xvfb kdelibs4c2a libkonq4

Hit ‘y’ to solve dependencies!

Now, get khtml2png from http://sourceforge.net/projects/khtml2png/ and install it:

dpkg -i khtml2png_2.7.6_i386.deb

Then, start your ‘fake’ X server:

/usr/bin/Xvfb :2 -screen 0 1920x1200x24

Of course, you may reduce the resolution to your needs. But remember the display number (:2) you set for Xvfb.

And finally, you may use khtml2png to fetch any website you like:

/usr/bin/khtml2png2 --display :2 --width 1024 --height 768 http://www.thomasgericke.de/ /tmp/website.png

Don’t worry about the fact that the package is named khtml2png and the binary is called khtml2png2. It’s okay!

I have a little magical wrapper around that stuff which gets URLs out of a database and performs some checks. Images are save with wget and converted to PNG, websites are fetched with khtml2png. Both are saved and thumbnailed on-the-fly with PHP.

I call khtml2png via cron like this:

/usr/bin/khtml2png2   --display :2 \
                      --width 1024 \
                      --height 768 \
                      --time 42 \
                      --disable-js \
                      --disable-java \
                      --disable-plugins \
                      --disable-redirect \
                      --disable-popupkiller \
                      http://www.thomasgericke.de/ \
                      /tmp/website.png

My script is started every minute and checks if new URLs have to be fetched. It also checks if existing PNGs are older than 24 hours and, if so, the URL will be fetched and the PNG overwritten.

Just let me know, if you have any further questions.

Hi there!

http://unfake.it/ now provides a tiny JavaScript bookmarklet which may be saved in your browser. While surfing on the web, you can click this bookmark and the page you’re currently viewing will automatically be faked/shortened.

But the real new feature is the preview function: by adding an asterisk ( * ) at the end of any faked URL, you’ll get a preview page which shows an image of the final destination. And this is really magical! Destination URLs may either be pages or images and they are initially fetched once a minute and refreshed once day. Preview of images will — of course — remain untouched. Whole websites are magically grabed as image and displayed as thumbnails.

An example?

http://unfake.it/XQh*

Check it out, this is really cool.

Since I lead a team of highly specialized IT personnel, who are mostly certified LPI level 1, level 2 or even level 3, I decided it was my turn to give it a try. I passed LPI 101 (part one of LPIC-1) on 2009-01-28 and LPI 102 (part two of LPIC-1) on 2009-02-02.

Here’s what you might wanna know about how to get LPIC-1 certified:

I did not learn a lot, because I’m used to Unix administration since about 1995. But I can highly recommend to buy and read the following book: LPIC-1 by Peer Heinlein. If you’re experienced, most parts may sound very familiar, but do you know all switches and all parameters of certain commands? Do you know, which commands and what paramaters LPI might ask? No? I guessed.

Here’s (in a very brief abstract), what you should know before trying to get certified:

1. Hardware & Architecture
   • Do you know what interrupts are and where information are stored during the system’s runtime?
   • What is IO?
   • Do you know how to configure PCI expansion cards manually?
   • How to connect and eventually configure a USB device?
   • What is a Winmodem?
   • Can you configure serial settings for a modem or a sound card?
   • Do you know the difference between SATA, SCSI or even external USB drives?
   • How does the system connect, name and use them?
2. Linux Installation & Package Management
   • Do you know how to set up partitions?
   • And why exactly this way?
   • Why can some directories be on its own partition? Why should some be? Why must some not be?
   • You can install and use LILO and Grub?
   • What are the differences?
   • Can you unpack, compile and install program sources?
   • You know what shared libraries are and why they exist?
   • Do you know the difference between RedHat and Debian package management?
   • Do you know the most important switches and parameters (nearly all of them) of those packaging tools?
3. GNU & Unix Commands
   • Do you know what a shell environment is?
   • When and how will an environment be inherited?
   • Can you set and unset variables?
   • Do you know (including the usage) of all of the following tools? cat, cut, expand, fmt, head, hexdump, join, nl, paste, pr, sed, sort, split, tac, tail, tr, unexpand, uniq, wc. If not, get used to them!
   • Do you know how to create, delete, read, move and copy files?
   • Have you understood what STDIN, STDOUT and STDERR are and how they may be (re)directed?
   • Can you kill processes, stop them, send them into the background or get them back to the foreground?
   • Do you know what the process priority means and how it can be manipulated?
   • Do you know how regular expressions work?
   • Do you have problems using vi? If so, get used to this editor!
4. Devices, Linux Filesystems, Filesystem Hierarchy Standard
   • Do you know how to set up a partition manually?
   • Even a swap partition?
   • Can you add new swap space while your system is running?
   • Can you check, tune and repair a filesystem?
   • Do you know when and how partitions are mounted? And why?
   • Can you set up and use quotas?
   • Can you get and read quota reports?
   • Do you know what file and folder permissions are? And how they usually should be?
   • Can you modify them?
   • Do you know what special bits (files/folders) are?
   • Can you explain the difference between hard and soft links? Can you create, use and delete both of them?
   • Do you know how find, which and locate work? How and why?
5. The X Window System
   • Do you know where the X configuration is stored?
   • Have you understood the difference between a xserver, a window manager and a display manager?
   • Can you redirect the graphical output of a program onto another machine?
   • Do you know in which part of the xserver’s configuration fonts are defined?
6. Kernel
   • Do you know what kernel modules are?
   • How can you query them?
   • How do you add or remove them?
   • Do you know how to reconfigure a kernel?
   • And how to compile and install it?
   • What about LILO and Grub? Anything to do after you installed a new kernel? Why? Why not?
7. Boot, Initialization, Shutdown and Runlevels
   • What are runlevels?
   • Where are they defined?
   • What about logfiles during a system boot?
   • Do you know how to shut down a system safely? Even if still users are logged in?
8. Printing
   • (Just my opinion: printing has been a freakin’ show ever since!)
   • What printing systems do you know?
   • What commands are used to print or to (re)configure the printing queue?
   • Do you know what happens to you print job after you startet it?
   • Do you know when (and when not) it has to be converted? And how this happens?
   • Can you set up a remote printer?
   • Even a remote printer on a Windows system?
9. Documentation
   • Do you know the different type of man pages?
   • And have you ever heared of whatis or apropos?
   • Do you know which messages users get before or after the log on to a system? And why?
10. Shells, Scripting, Programming and Compiling
    • Do you know all BASH fundamentals including profile, variables, environment or built-in functions?
    • Can you write your own (simple) BASH scripts?
    • Can you let your script perform a loop?
    • Do you know how to compare strings or expressions (using test or if [ …])?
11. Administrative Tasks
    • Do you (exactly!) know what a shadow system is? What is it? How does it work? And why is it used?
    • Can you add, remove or reconfigure users or groups using tools?
    • And can you even do it right within the config files?
    • Do you know what the skeleton is used for?
    • Do you know what syslog is used for and how it works?
    • Can you read and/or rotate logfiles?
    • Even if they are stil locked by a process?
    • Do you know how to set up certian jobs which should be started once in the future or even frequently?
    • Do you exactly know the syntax of a cronjob?
    • Do you know how to make backups? How to store them and how to restore and use them?
    • Can you set the correct time and date on your system?
    • Even for the hardware clock?
12. Networking Fundamentals
    • Have you understood what an IP address is and how it looks like?
    • Can you calculate the netmask and broadcast of a network?
    • Do you know what IP, TCP, UDP, ICMP mean? And the difference?
    • Do you know the ports of the most common services?
    • Can you change the IP address of a running system?
    • Could you debug TCP connection failures?
    • Do you know what routing means and when it is needed?
    • Can you configure a modem as your PPP device?
13. Networking Services
    • Do you know how incoming connections are handled?
    • What is the difference between inetd and xinetd?
    • How could you migrate?
    • Can you block or allow certain connects using simple files under /etc/?
    • Do you know how mail aliases are handled?
    • And what if they have changed?
    • Can you start, stop, restart and even configure an Apache webserver?
    • Do you know how to mount or export filesystems using NFS?
    • Can you mount a samba share?
    • Do you know how to configure your system using DNS servers? How?
    • Can you maintain a SSH server?
14. Security
    • Do you know how incoming packets are handled by the kernel?
    • Can you add new iptables rules?
    • What are sockets?
    • Do you know netstat?
    • Do you know how to secure your system using xinetd, iptables, syslog, etc.?
    • Can you block logins to your machine using a single file?
    • Do you know how to secure access to files using permissions?
    • Do you know what the umask is?

This is wow, isn’t it?!

If you’re using a linux/unix system for quite a while, most of the questions should sound easy. But, indeed, nearly every tool has switches and/or parameters, you’ve never ever heard of. So, I highly suggest, you should read at least the man pages of every program, tool or task I mentioned above.

If you do know the (of course right) answer for every question, you should have no problems passing the tests for LPIC-1 certification.