HIGH COMP MEMORY % Comp ( above 90 )
==================================
Depending on your AIX version and tunable settings you may start paging out computational content - which usually slows down your corresponding application as AIX is paging in that case entire processes to disk - it stops the process, moves it - and than continues using it.
When you are that high in comp memory, that means you have literally no room in memory for non-comp (file caching) what is as well vital for many applications. If they are requesting memory, than the system starts thrashing (excessive scanning / freeing memory) what may cause a huge amount of cpu overhead and slowdown of your box - this can go that far that no other processing happens on the corresponding server.
Since you are already starting to page (2% paging space used) you are at risk that when you filling your paging space, your system might at first randomly kill processes to survive and later either hang with no login possible or crash and restart itself to free up memory.
What you need to do is monitor your system very closely - like if the amount of comp memory in use is growing and if your paging space utilization is growing.
IBM recommends to have comp memory around 75% (and from a performance perspective I can only confirm this - my boxes - DB servers - perform best when they are somewhere between 66 and 80%) - depnding on the criticality of your applications, buying a few gig of memory may be cheaper than risking a crash.
I think you need to understand what computational memory is - and what computational memory NOT is to understand why it is important to have sufficient non-computational memory in your system?
Computational memory = basically everything that does not exist on your hdisk - for example DB content changed but not yet committed, anything volatile that is work in progress - basically everything that will be gone when your system crashes.
Non computational memory is everything that has a place on disk but still needs to be working in memory - like shared libraries, executables, ... on top of this each and every disk IO needs 1 page of non-computational memory - so every read and every write.
Usually your system is hopefully tuned in a way that computational memory is NOT paged to disk before you reach 97% (minperm=3%). As well your lrud_file_repage is hopefully set to 0. That means until you do not exceed 97% computational memory, your system will page when it needs to fulfill IO or filecaching needs, or if you start a process or you carry out a job (batches, backups and much more) - but it will not page out for example the DB processes. Nonetheless it will fill up your pagingspace.
If your computational memory needs exceed 97% than the system does not care anymore if what is in memory and needs to be paged out is your database- or application data or if its filecaching - it will just page out everything.
You are saying you are running oracle. Each oracle process doing work can consume 1 pp worth of (shared) memory. In our company that is about 64 mb as we have rather small disks - but this can be 128, 256 or more MB in certain setups - and oracle can have quite a few processes (some of our DB servers have a few thousand). Oracle is a 64bit application, so shared memory is only limited by the amount of physical memory available in the box - so this should not cause application issues - BUT Oracle has on top of its processes as well its SGA - a fixed amount of memory that is allocated for oracle only - so neither the kernel nor any other application running on your system will have any access to it. And - if the SGA is not locked, it may potentially be paged out in total to pagingspace - if this happens, it is kind of disatrous for your DB performance.
On top of that you are running Tuxedo (weblogic) and websphere on your server. I do not know for sure but I believe that Tuxedo is a 32 bit application - I do know for sure that Websphere is. And both are preallocating predefined additional shared memory for themselves. Problem with 32bit applications is, that they cannot use unlimited shared memory - AIX offers only 13 shared memory segments for their use (this can be amended with certain system settings to be exported in the environment but I would not assume that has been done in first instance). And ... 32bit applications use different shared libraries than 64bit applications, so the system needs to hold even more content of this kind in (non-computational) memory - which makes your non-comp needs even bigger. Generally its not such a great idea to mix DBs and Webapplications on the same server for various reasons - the above is one of many, the different needs in workload (DBs = large parallel serial threads, Webapplications lots of mini-threads and random with corresponding loads of IOs ) ...
If you like, you can post a vmstat -Iwt 2 10 output taken at a busy timeframe and vmstat -s output and maybe svmon -G here ... that will help me explain a little further ?
Interesting to see would be as well the top area of commands like svmon -U oracle and similar for your other application users assuming that Tuxedo and Websphere are running under their own ID ? You will be surprised how much that adds up.
=============================================================