You need to put it in the startup script for the application server. The details depend on the server, In Tomcat 5.0, you will normally create a script like this:
@echo off
setlocal
set JAVA_OPTS=-Xms256m -Xmx256m
%TOMCAT_HOME%\bin\startup
endlocal
This one is for Windows. It sets additional parameters for the java call that starts Tomcat, and calls the bin/startup.bat that actually starts the server.
To see what parameters you can use, and how it works, have a look at the Tomcat bin/startup.bat and bin/catalina.bat.
For other appservers the process is similar: There is some mechanism for adding params to the appserver startup, find it and add your stuff.