VIM as IDE

I have used a fair few IDE's
I can't get past VIM.

I decided to really start to use vim.

here is a copy of my vim setup

http://code.google.com/p/vim-config-tranter/


here is a tute i followed to set up what i liked about it
http://taggedzi.com/articles/display/vim-for-php-developers-part-3

I love VIM.

Fun with Squid and Python

Viewing images that users of squid have been viewing.

The goal of this is to have some fun with your squid server using a python script.
the idea is you have an access log which contains all your logs from squid.
This is usually located at
/var/log/squid/access.log
*may differ on different OS's*

this contains a list of URLS that have been visited and by which IP Address.

This script goes through this access.log and creates an html file which contains images from the access log.

Code:

http://code.google.com/p/squidimage

Click Here  to download src code.

Download the Source code from above.

Simply run by 

python main.py -i /var/log/squid/access.log -o output.html

You can have a more advanced scoped output

python main.py -i /var/log/squid/access.log -o output.html -x <amount> -a <ip>

Full Example

python main.py -i /var/log/squid/access.log -o output.html -x 200 -a 10.0.0.140

More?

python main.py -h

Output Example:

Now, you can what images people have been looking at.

UPDATE:
The next version of squidimage you will be able to provide a min size of images to load into the HTML file.

Transparent Proxy using squid and linux

Step 1) -Intro

The goal of this is to route all traffic through a transparent squid proxy.

What this means, is all internet traffic will go through your server first instead of your default router.

The benefits:

• Can cache web pages via squid and therefore have faster browser
• Can be a snoop and look at what people are looking at
• Have more control over your flow of which sites are blocked and which are not. 

 This method of routing traffic through a server with only one Ethernet port can be bypassed quite easy.

Currently:

When a new device connects to your network, your router will provide its IP Address via the built in DHCP server within,

Along with this IP Address, it will provide itself as the Default Gateway and the DNS.

In this Example.

Router:10.0.0.138

Server:10.0.0.140

Therefore if a new device connected it would get

IP:10.0.0.xxx

Default Gateway:10.0.0.138

DNS:10.0.0.138

After:

The DHCP Server within the router will be turned off, and the DHCP server within your linux box will be enabled.

It will be configured to give

IP:10.0.0.xxx

Default Gateway:10.0.0.140

DNS:10.0.0.138

Notice the Default Gateway is now set to the Server.

MAKE SURE YOU CHANGE ALL IP ADDRESSES AND ROUTER ADDRESSES TO SUIT YOUR NETWORK OTHERWISE NOTHING WILL WORK.

Step 2) -Software needed

Install Squid - Proxy Server

Debian Based Linux:

sudo apt-get install squid

Redhat/Fedora/Suse/Centos Based Linux:

s
yum install squid

Install dhcpd - DHCP Server

Debian Based Linux:

sudo apt-get install isc-dhcp-server

Redhat/Fedora/Suse/Centos Based Linux:

su
yum install dhcp

Step 3) - DHCP config 

#as root
vim /etc/dhcp/dhcpd.conf

and then paste the following

default-lease-time 600;
max-lease-time 7200;
option domain-name-servers 10.0.0.138;
option routers 10.0.0.140;

subnet 10.0.0.0 netmask 255.255.255.0 {
        option domain-name-servers 10.0.0.140;
        range 10.0.0.1 10.0.0.100;
        option broadcast-address 10.0.0.255;
        option routers 10.0.0.140;
        }

Step 4) - Iptables config

Because the DHCP Server is setting the Default Gateway to the server, it has to route the traffic for port 80 to the port of your Squid server.
It also reroutes the traffic on port 443 (https/SSL) back to your router.

create script for iptables

#as root
vim /usr/local/bin/redirecttraffic

and then paste the following

#!/bin/sh

# Squid server IP
SQUID_SERVER="10.0.0.140"
LOCAL_ROUTER="10.0.0.138"

# Interface connected to Internet
INTERNET="eth1"

# Address connected to LAN
LOCAL="10.0.0.0/24"
# Squid port
SQUID_PORT="3128"

# Clean old firewall
iptables -F
iptables -X
iptables -t nat -F
iptables -t nat -X
iptables -t mangle -F
iptables -t mangle -X

# Enable Forwarding
echo 1 > /proc/sys/net/ipv4/ip_forward

# Setting default filter policy
iptables -P INPUT DROP
iptables -P OUTPUT ACCEPT

# Unlimited access to loop back
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT

# Allow UDP, DNS and Passive FTP
iptables -A INPUT -i $INTERNET -m state --state ESTABLISHED,RELATED -j ACCEPT

# set this system as a router for Rest of LAN
iptables -t nat -A POSTROUTING -o $INTERNET -j MASQUERADE
iptables -A FORWARD -s $LOCAL -j ACCEPT

# unlimited access to LAN
iptables -A INPUT -s $LOCAL -j ACCEPT
iptables -A OUTPUT -s $LOCAL -j ACCEPT

# DNAT port 80 request comming from LAN systems to squid 3128 ($SQUID_PORT) aka transparent proxy
iptables -t nat -A PREROUTING -s $LOCAL -p tcp --dport 80 -j DNAT --to $SQUID_SERVER:$SQUID_PORT

# if it is same system
iptables -t nat -A PREROUTING -i $INTERNET -p tcp --dport 80 -j REDIRECT --to-port $SQUID_PORT

#forward ssl over to local router
iptables -t filter -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -t filter -A FORWARD -i $INTERNET -p tcp --dport 443 -j ACCEPT

iptables -t nat -A POSTROUTING -o $INTERNET -j SNAT --to-source $LOCAL_ROUTER


#open everything
iptables -A INPUT -i $INTERNET -j ACCEPT
iptables -A OUTPUT -o $INTERNET  -j ACCEPT

# DROP everything and Log it
iptables -A INPUT -j LOG
iptables -A INPUT -j DROP
                             

Step 5) - Squid Config

squid config

#as root
#save old squid.conf
cd /etc/squid/
mv squid.conf squid.conf.default
vim squid.conf

and then paste the following

# Recommended minimum configuration:
#
acl manager proto cache_object
acl localhost src 127.0.0.1/32 ::1
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32 ::1

# Example rule allowing access from your local networks.
# Adapt to list your (internal) IP networks from where browsing
# should be allowed
acl localnet src 10.0.0.0/24    # RFC1918 possible internal network

acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl CONNECT method CONNECT

#
# Recommended minimum Access Permission configuration:
#
# Only allow cachemgr access from localhost
http_access allow manager localhost
http_access deny manager

# Deny requests to certain unsafe ports
http_access deny !Safe_ports

# Deny CONNECT to other than secure SSL ports
http_access deny CONNECT !SSL_ports

# We strongly recommend the following be uncommented to protect innocent
# web applications running on the proxy server who think the only
# one who can access services on "localhost" is a local user
#http_access deny to_localhost

#
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#

# Example rule allowing access from your local networks.
# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow# We recommend you to use at least the following line.
hierarchy_stoplist cgi-bin ?

# Uncomment and adjust the following to add a disk cache directory.
#cache_dir ufs /var/spool/squid 100 16 256

# Leave coredumps in the first cache dir
coredump_dir /var/spool/squid

# Add any of your own refresh_pattern entries above these.
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
 localnet
http_access allow localhost

# And finally deny all other access to this proxy
http_access deny all

# Squid normally listens to port 3128
http_port 3128 transparent

Step 5) - Final

Don't forget to configure your router to not issue ip addresses.

Final step is making the ipconfigs script run on startup

There is many ways of doing this.

su 
#make script executable
chmod +x /usr/local/bin/redirecttraffc

vim /etc/rc.local

#paste in
./usr/local/bin/redirecttraffc

NOW REBOOT ALL YOUR MACHINES INCLUDING YOUR SERVER.