CCIE Lab using GNS3 and Quad Nics for Switches

There are 3 different methods how you can connect your real switches to your GNS3 routers:

• Using a breakout switch.
• Using cheap USB network adapters.
• Using Quad NIC PCI cards.

 

This article is how you can build your own GNS3 server using Quad NICs, if you are looking for a serious and reliable CCIE lab this is the way to go. I built the breakout switch before which you can see in my other article but the Quad NIC solution seems to be far more stable.

Before I continue, if you don't like reading you can also watch my Youtube video which you can find at the bottom of this article!

Equipment List:

• 1x Codegen 4U-500 19 inch Rack: I like this case because it's cheap, sturdy and in 19 inch format. If you want you can put this case and the switches in a nice 19 inch rack.
• 1x Asus P8H67-V Motherboard: I bought this one because it has video onboard, 3 PCI slots and 1 PCI-express slot for my serial ports to connect the switches to.
• 1x 8GB DDR3 Corsair memory: In the future I want to run virtual machines on this server as well, just for GNS3 it's overkill.
• 1x Intel I5 CPU: Same story...for GNS3 a bit too much but if you want to run virtual machines as well it's nice.
• 1x Corsair Powersupply: Just because I need one, 500 watt should be sufficient.
• 3x D-Link DFE 580TX: These are awesome, 4 FastEthernet ports per PCI-card and not too expensive on Ebay.
• 1x 4-port serial PCI-express card: Very nice card which has 4 serial ports to connect my switches to ( you can also buy 4x USB 2 Serial instead).
• 2x Cisco 3560 Switch.
• 2x Cisco 3550 Switch.

 

If you are on a low budget just get cheaper hardware. A core 2 duo can run 12 routers with no problem and you don't need more than 4GB of memory. I wouldn't recommend getting other Quad NICs than the D-Link DFE580TX since it's working right out of the box and not too expensive. You can also use 4x 3550's but keep in mind the QoS of the 3560 is different and there is no way to practice private vlans.

Installing Ubuntu:

Installation of Ubuntu was a piece of cake, the D-Link Quad NICs and the serial port PCI-express card are detected and work out of the box, I didn't have to install any device drivers. If you install GNS3 from the Ubuntu repositories you'll have GNS3 0.73 which is working fine for me, there are no features in GNS3 0.74 that I need for this CCIE lab. I don't have a monitor or keyboard connected to this machine so I decided to configure VNC server which is installed by default in Ubuntu, just configure it and you are ready to go.

Installing ser2net:

Ser2net will make sure you can access your serial ports from the network by telnetting into them, this is what you need to do:

First we will install ser2net, this application will make sure you can access your serial ports by using telnet:

sudo apt-get install ser2net

The next step is to find out which serial ports (tty in linux) are connected to your switch:

dmesg | grep tty

This will tell you which tty lines are in use, for me this was ttyS4, ttyS5, ttyS6 and ttyS7.

Now we need to edit our ser2net configuration file and set the correct ports:

sudo vi /etc/ser2net.conf

This is what my configuration looks like:

4000:telnet:0:/dev/ttyS4:9600 8DATABITS NONE 1STOPBIT banner
4001:telnet:0:/dev/ttyS5:9600 8DATABITS NONE 1STOPBIT banner
4002:telnet:0:/dev/ttyS6:9600 8DATABITS NONE 1STOPBIT banner
4003:telnet:0:/dev/ttyS7:9600 8DATABITS NONE 1STOPBIT banner

If you telnet to port 4000 it will access ttyS4, port 4001 for ttyS5 and so on. By default it will look like "4000:telnet:600" but I changed it to "4000:telnet:0" so there is no timeout.

Before this changes apply you need to restart the service:

sudo service ser2net restart

Excellent that's all you have to do for ser2net, now if you want to telnet into your serial ports this is what you do:

telnet  

And you will have access to your serial port.

Remote access to your GNS3 routers:

If you want remote access to your GNS3 routers it's possible to telnet directly to the IP address and the correct port number. If you don't know the port number you can look them up by running GNS3 and typing in "list" in the console at the bottom.

Here you can see that these 3 routers are using port 2000, 2001 and 2003. You can directly telnet into them and you will have access. I highly recommend not doing this because if you lose your connection there is no way to get back in the console and you will be locked out. It's better to use the "screen" command. Screen will let you open a session on a linux machine which will keep running in the background even when you are disconnected. This is how you use it:

First use SSH to login to your GNS3 server:

ssh 

Once you are logged in and see the command-line you can use screen to telnet into a router:

screen -S R1 telnet localhost 2000

This command will start a new "screen" and telnet into port 2000, in my example this is router1.

If your connection fails, just login the GNS3 server using SSH again and check the detached screens:

screen -ls

This will give you an output like this:

10757.R1    (06/18/2011 08:01:15 PM)    (Detached)
10754.R2    (06/18/2011 08:01:12 PM)    (Detached)

Now you can resume your session by using this command:

screen -r R1

And you are back at the console of your router!

That's all you need to do to build your own CCIE lab using GNS3 and some real switches. The only thing I still could do is build some scripts that will automatically open all my tabs and ssh / telnet into the devices...you don't really need it but it's fun and will save you some time when starting up the lab.

This solution works far better for me than using the breakout switch.

I had some trouble with flapping mac addresses, interfaces on the breakout switch going in err-disabled mode and CDP is troublesome on a cisco 3550 breakout switch. This is working 100% flawless so far!

What do you think? If you have any questions or comments please let me know by leaving a message below.

Update:

Recently I have been running into some trouble with my GNS3 server which I believed was something from the past...IRQ conflicts! The D-link quad nics support IRQ sharing so this is no problem at all. My cheap 4-serial port pci-express card however requires an IRQ per port. Since it was on the same IRQ as one of my Ethernet interfaces the machine decided to crash/become unreachable. Keep this in mind when you are buying hardware, check if it supports IRQ sharing or not.

I fixed this problem by getting rid of the cheap 4-serial port pci express card and I am now using my Cisco router as terminal server for the 4 switches.

I also replaced my motherboard since the one from Asus was causing issues with the PCI cards. I replaced it with a MSI motherboard:

MSI Z68A-G43