You can run any of the code you create on your development machine. That's we J2EE developers develop this stuff.
Unless of course, you want the public to see it. Then you will either need to host the application on a publicly visible server or pay to have it hosted.
For learning, I'd stick to running these things on your development machine.
No.
It depends on what you are trying to develop/host. For servlets and jsp's, you can just use jakarta tomcat. If you want to do EJB development, you will need a full blown application server. There are many application servers freely available for development use (e.g. JBoss, HP-AS). Most application servers either natively support or come bundled with a web engine to serve up static content so you don't need a web server like IIS or apache.