Wednesday

Oracle Errors

ORA-25153
While applying a MP through Change Assistant, a dms script failed giving below error-

Import  PSPROJECTDEFN  1
SQL Error. Error Position: 0  Return: 25153 - ORA-25153: Temporary Tablespace is Empty ORA-06512: at "SYS.DBMS_STATS", line 20337 ORA-06512: at "SYS.DBMS_STATS", line 20360 ORA-06512: at line 1
RECSTATS PSPROJECTDEFN LOW
Error: SQL execute error for %UpdateStats(PSPROJECTDEFN)
Error: Import Record name PSPROJECTDEFN



Troubleshooting  -

SQL> select tablespace_name,file_name from dba_temp_files;
no rows selected

SQL> select name from v$tempfile;
no rows selected

SQL> select property_value from database_properties where property_name like '%D
EFAULT_TEMP_TABLESPACE%';
PROPERTY_VALUE
--------------------------------------------------------------------------------
TEMP


The sql lists the default temporary tablespace of database - TEMP. It is created during database creation process (createdb10  DEFAULT TEMPORARY TABLESPACE TEMP TEMPFILE '... ). I verified the corresponding data file - xxx/temp01.dbf and it was already there.


Solution -

SQL> ALTER TABLESPACE temp ADD TEMPFILE 'xxx\temp01.dbf' SIZE 20
M REUSE AUTOEXTEND ON NEXT  640K MAXSIZE UNLIMITED;

SQL> select tablespace_name,file_name from dba_temp_files;
TABLESPACE_NAME
------------------------------
FILE_NAME
--------------------------------------------------------------------------------
TEMP
P:\xxx\TEMP01.DBF


Next was to manually import - PSPROJECTDEFN through Datamover, and updating the statistics.

SQL>  exec dbms_stats.gather_table_stats('sysadm', 'PSPROJECTDEFN');
PL/SQL procedure successfully completed.

SQL> SELECT LAST_ANALYZED FROM DBA_TABLES WHERE TABLE_NAME='PSPROJECTDEFN';
LAST_ANAL
---------
17-JUL-13

RMAN, ASM and Data pump

Lets setup RMAN ( Recovery Manager) on oracle 11g database.

1) The recovery catalog must be created before RMAN can be used. Before creating catalog, create a tablespace, schema and user for recovery catalog. Grant CONNECT, RESOURCE, RECOVERY_CATALOG_OWNER to the schema owner.




2) Create catalog tables using RMAN. Register target database in recovery catalog, once registered, you can view target database tablespaces via REPORT SCHEMA command-
cd %ORACLE_HOME%/bin>connect CATALOG rman/rman@yourdbname
RMAN> CREATE CATALOG TABLESPACE <tblspcname>;
RMAN> REGISTER DATABASE;
RMAN> REPORT SCHEMA;

3) Now we can use RMAN to backup the database, lets specify the channel which tells oracle where to store the backup-
RMAN> CONFIGURE CHANNEL DEVICE TYPE DISK FORMAT 'c:/backup/ora_df%t_s%s_s%p';
RMAN> backup tablespace system;




Data Pump - Since Oracle 10g, Data Pump is an alternative to export/import utilities from earlier versions. It exports/imports - database tables, schemas or the FULL database.  To set up datapump to work in an environment-
1) Create directory - lets create a test directory on P: drive
SQL> CREATE OR REPLACE DIRECTORY test_dir AS 'P:\';
SQL> GRANT READ, WRITE ON DIRECTORY test_dir to SYSADM;
2) To Export/Import, cmd to ORACLE_HOME/bin and run expdp or impdp command as-

Table export/import-

expdp SYSADM/SYSADM@PSHR9150 tables=PSSTATUS directory=test_dir dumpfile=name.dmp logfile=expdpname.log;

impdp SYSADM/SYSADM@PSHR9150 tables=PSSTATUS directory=test_dir dumpfile=name.dmp logfile=expdpname.log;

Schema export/import-
expdp SYSADM/SYSADM@PSHR9150 schemas=SYSADM directory=test_dir dumpfile=SYSADM.dmp logfile=expdpname.log;

Full Database export/import
expdp SYSADM/SYSADM@PSHR9150 FULL=Y directory=test_dir dumpfile=SYSADM.dmp logfile=expdpname.log;



Network Exports/Imports - NETWORK_LINK parameter specifies a database link to be used as the source for the network export/import. It identifies the database link pointing to the source server and objects are exported from source and written to a directory on local server. EXP_FULL_DATABASE role need to be granted to both local and remote users.

* FLASHBACK_SCAN or FLASHBACK_TIME parameters are used in order to get the export consistent to the a point in time, as by default the exports using expdp are consistent on a table basis.

For help on export/import, use expdp help=y or impdp help=y


DBNEWID - this utility is provided since oracle 11g using which you can change the DBID of a copied database. As if you are working with RMAN, it distinguishes the databases based on DBID and once you copy a database, you can change its name to a new value (e.g. while refreshing one database from another) however the DBID is not changed and so you can not register the copied database to same RMAN repository.

Monday

How SSL is configured in Peoplesoft

How SSL is configured in Peoplesoft- Here are the steps to configure SSL -

1. Generate webserver' s private key and certificate signing request (CSR).
2. Submit CSR to your CA for signing-
3. Download the root certificate and intermediate CA certificates.
4. Download SSL certificate.
5. Import certificates into keystore.
6. Configuring the Oracle WebLogic Server to use the keystore.
7. Configure the Peoplesoft certificates.
 
1. Generate webserver's private key  and certificate signing request (CSR) -
1) Backup the existing pskey file-
    PS_HOME/<webdomain>/piaconfig/keystore$cp -p pskey pskey_ori
2) Change directory to piabin, create or change the keystore password by issuing the command-
 
$./pskeymanager.sh -create -storepass password
$./pskeymanager.sh -changekeystorepassword -new pshr9152 -storepass password
 
provide values for - 
-Specify an alias for this certificate,
-Specify a common name for this certificate.
-Answer to organization specific information.
-What key size would you like to use - 2048
-What key algorithm would you like to use - RSA 
-What key signing algotithm would you like to use -MD5withRSA
-Enter a private key password - <your_password>
 
Generating private key... 
.... 
Generating Certificate signing request 'CSR'.
Certificate signing request also written to <alias>-certreq.txt 
 
* The Certificate Signing Request (CSR) text file generated above should not contain any blank or trailing spaces.


2. Submit CSR to your CA for signing- Before you purchase an SSL Certificate, you need to generate a Certificate Signing Request (CSR) for the server where the certificate will be installed. To enroll for any of Symantec’s SSL Certificate services, you will need the following information:
- The length of time for the certificate,
- The number of servers hosting a single domain (up to 5 servers),
- The server platform
- An email address where Symantec can reach you to validate the information, and

- The Challenge Phrase - <your_password>


3. Download the root certificate and intermediate CA certificates - Download these certificates from your CA site and store within PS_HOME/webserv/<webdomain>/piabin/ as -
- rootca.cer,
- primary_inter.cer and
- secondary_inter.cer.

Root Certificate: A VeriSign Root Certificate is self signed x.509 certificate that includes the signature from VeriSign as the Certificate Authority which vouches for correctness of the data contained within the certificate. Root CA's are implicitly trusted. They are pre-installed into web browsers and many web servers.

Intermediate Certificates: In certificate hierarchy, a subordinate Intermediate CA certificate will be issued by the Root CA to issue end-entity SSL certificate. This creates a chain of trust that begins at the Root CA, through the Intermediate CA and ending with the SSL certificate.



4. Download SSL certificate - Your CA will email you your certificate or download it from the designated server link. Save it as Cert.cer
 5. Import certificates into keystore -

Step 1 – Install Root Certificate and intermediate Certificates-
$./pskeymanager.sh -import -alias RootCA -storepass password -file rootca.cer
$./pskeymanager.sh -import -alias primaryintermedicate -storepass password -file primary_inter.cer
$./pskeymanager.sh -import -alias secondaryintermediate -storepass password -file secondary_inter.cer

Note: To delete an imported certificate, use below command-
./pskeymanager.sh -delete -alias secondaryintermediate -storepass password

Step 2 – install your SSL Certificate. Copy the SSL certificate to – PS_HOME/webserv/<webdomain>/piabin as cert.cer file.
$./pskeymanager.sh -import -alias <alias_name> -storepass password -file cert.cer

Configure the keystore for use in Weblogic -

1- On WebLogic server console, expand the Servers node and choose the server you will be configuring.
2- Next, go to Settings for PIA > Keystore (In Change Centre, activate Lock & Edit button)
3- Click on Change and choose Custom Identity and Custom Trust as your keystore configuration type, then click Save.
4- The "Custom Identity Keystore PassPhrase" should be the password you specified when the keystore was created. [password]
5- You will again be asked to enter your keystore password and confirm.
6- Enter and confirm Custom Trust Keystore Passphrase: password
7- Click Save.
8- Next, go to Settings for PIA > SSL.
9- Specify the Private Key Alias and Passphrase that were used when creating your keystore.
10- Click Save, and go to Advanced
11- Choose Hostname Verification as None and Save.
12- Click on Activate Changes.
13- Reboot the WebLogic server. Your keystore should now be installed and enabled.
14- Verify the SSL.
 

Sunday

Peoplesoft Installation on windows server.

Installing the Oracle database software - The Oracle Universal Installer prompts you to provide it the path of the oracle base. Oracle Base and Oracle Home are directory locations and are recommended to be set as defined in OFA (Oracle Flexible Architecture) standards. This helps a DBA to effectively manage the system and also, improves performance by minimizing disk contention.

ORACLE_BASE - c:\oracle\product\11.2.0

ORACLE_HOME - c:\oracle\product\11.2.0\dbhome_1

Once the database software has been successfully installed, lets start with creating an oracle instance/service and password file.


Create pfile (initPSHR9152.ora) and startup database in nomount stage with spfile.


Now install the peoplesoft PeopleTools and Application -

Extract downloaded compressed folders, and run PeopleSoft Installer (setup.exe) from disk1. Provide License Code, select Non-Unicode or Unicode database type radio button, and specify PS_HOME location, Connectivity Program Directory location (C:\oracle\product\11.2.0\dbhome_1\BIN). Provide path to disk2 and disk3.






Installing PeopleTools
Installing Oracle Tuxedo
Installing Oracle WebLogic
The Pure Internet Architecture

Saturday


CPU Fixes - Oracle releases a critical patch update CPU as a collection of patches (security fixes) which are usually cumulative and these are recommended to applied as soon as they are made available.

How to check the current database version -
SELECT * FROM V$VERSION;

Out-of-place path set upgrade - the recommended best practice where you install the patch set into a new separate oracle home. The patch sets are full installation of oracle database software.

In-Place Upgrade -

Friday

PeopleSoft Upgrade

A typical upgrade process comprises of below steps-
 
1) Evaluate new release, and decide to or not to upgrade.
2) Plan for upgrade,
3) Install new release,
4) Perform Initial Upgrade Pass,
5) Test/Adjustments,
6) Test Move to Production,
7) Test
8) Is Result Acceptable? - if no then make adjustments and go back to step 5)
9) Schedule upgrade cutover date,
10) Perform Final Move to Production (MTP),
11) Final Acceptance Test,
12) Is ready to Go Love? if not then make final adjustments and repeat from step 9)
13) Go Live on new release
 
 
Upgrade Process - migrating objects from one database to another and/or updating other system components e.g. software and/or hardware.
 
Database objects as well as software determine the release level. An upgrade may or may not installing new executable files. During upgrade, one or both of below two tasks are performed in order-
 
1- Update software,
2- Copy any new objects into database and adjust all system components accordingly.
 
There are 3 main system components that can be affected by or involved in an upgrade-
 
1) Environment - Hardware and Software including client and server machines, client and server OS, RDBMS software and database and network software.
2) PeopleTools - Software programs, utility scripts and reports, database tables and data that provide framework for PS applications.
3) Applications - object definitions. 
 
The major phases of an upgrade project are -
1) Initial Pass,
2) Test Move to Production Pass, and
3) Final Move to Production Pass.
 
 
1- Initial Pass - the main steps in this pass are -
1) Create a copy of production,
2) Install Demo for new release,
3) Compare and Analyse, here Demo is the source for all new release objects and system data and a copy of production is the target.
4) Once compare analysis is complete, new release is incorporated into copy of production to create first "upgraded copy of production".
 
2- Test Move to Production Pass - use above created "upgraded copy of production" as source and "a recent copy of production" as target and in iterations, every time and after each pass, a new updated "new copy of production" is created based on most recent production data for each pass.
 
3- Final Move to Production - Freeze production database. In this final move, the most recent successful output from step-2 is the source and production is the target.
 
 
 The important tasks carried out during an upgrade project are -
1) Determining upgrade path - determine supported upgrade path, verify whether you can directly upgrade to target release or you need to upgrade to a previous release before moving to target release. Evaluate complexity of upgrade efforts based on number of modules implemented, number of customizations number of interfaces and number of integrations. Determine costs with each aspect of upgrade.
2) Upgrade planning - treat upgrade as a formal project, constitute an upgrade team, decide whether to change or add a business process and prepare a Go Live check list.
3) Evaluate architecture - platform, middleware, non-production hardware ? etc.
4) Calculate sizing and Identify custom code, scripts and interfaces - customizations and custom reports will require extensive testing.
5) Install new release- install verification tests, and get code current by applying the most recent tools and application patches.
6) Technical Upgrade & Retrofitting - identify data migration tasks and minimize application data to upgrade. Test with a copy of production and continue with most current data.
7) Post upgrade activities - end user trainings, functional validation of the system, UAT and Go Live.

Version numbering and Caching.

PSVERSION table in peoplesoft stores an acronym for each object type e.g. RDM (Record Data Management), UPM(User Profile Management) etc. It is updated by the panel processor (component processor) and every time the peoplesoft component processor attempts to retrieve an object definition, it first checks the PSVERSION table which provides the most updated version of that specific object type. Whenever an object definition is changed, this table is updated i.e. if a record definition is updated through application designer, its version in PSRECDEFN is first updated and afterwards the application updates the PSVERSION table.

The application server compares the version number from the OBJTYPE.KEY file (e.g. RDM.KEY cache file) to that in the PSVERSION. If PSVERSION.VERSION is greater than that of IDX (the .KEY cache file) then it will retrieve object from database and OBJTYPE.DAT (e.g. RDM.DAT cache file) will be updated with the latest version of the object.

The PSxxxDEFN.VERSION must be less or equal to PSVERSION.VERSION for the corresponding object type name in order to application server's caching algorithm to work. To repair application server caching, we run VERSION AE program.

Generally, PSVERSION and PSLOCK, both the tables should have to be in sync, however the discrepancies are resolved by running the VERSION application engine from command line. It resets version number on all definition tables, PSVERSION and PSLOCK.

* Clear cache for two tier clients after running the VERSION app engine.

Application Server Caching - caching enables the application server to store object definitions locally on app server, thus eliminating unnecessary trips to database server. If caching were disabled, the system would need to retrieve definitions from database tables every time for each request which would add significant overhead to each transaction and adversely affect system response time.

Peoplesoft provides two types of app server caching -

1) Non-Shared Cache - enabled by default. In this mode, each server process maintains its own separate cache file.
2) Shared Cache - to enable this mode, set ServerCacheMode=1. In this configuration, system assumes that a pre-loaded cache exists. When application server is booted and it finds no cache, then it reverts to non-shared cache mode.


* The delivered application engine program- LOADCACHE preloads the cache. Before loading the shared cache, run SYSAUDIT report and make sure that it is clean otherwise cache loading may fail.

Thursday

Oracle database installation on Linux

Here are the steps to install Oracle database on Linux platform. I have installed this on Red Hat Enterprise Linux 4.

Preparing system for install-
1) Login as root user and check memory and swap space, /tmp space ($df -k /tmp)
2) Create oracle user account.
 


3) Run Installer program -



You see this error- Installer cannot continue installation if user is "root". OK, let switch user to oracle and continue. Still the error- "... refused by server"


Now disable the access control, and run the installer.

 
The Oracle Universal Installer is now ready for installation.
 
 

Shell Scripting

Writing a series of command for the shell to execute is called Shell Scripting. It can combine lengthy and repetitive sequences of commands into a single and simple script, which can be stored and executed anytime. This reduces the effort required by the end user.

Create a file using a vi editor(or any other editor).  Name  script file with extension .sh. Start the script with #! /bin/sh
Write some code.
Save the script file as filename.sh
For executing the script - $filename.sh

“#!” is an operator called shebang which directs the script to the interpreter location. So, if we use“#! /bin/sh” the script gets directed to the bourne-shell.


#comment - the syntax to add a comment.


Shell Variables- There are some special predefined variables available when a shell script is running.

$0 - name of currently running script
$# - number of arguments given to this script
$1, $2...$9 - $n is nth argument, and it is undefined if there is no nth argument.
$$ - PID of the script.

Wednesday

A Peoplesoft Database Refresh


For example, here the source database will be PRD, and the target database will be the TST database that you're refreshing from source database. These are exhaustive steps covering most of the tasks.

 

Prepare for refresh -

-Check and compare the tablespace size, used space and free space for both the databases. (DBA_DATA_FILES, DBA_FREE_SPACE).

-Alter tablespaces and check adequate disk space; add datafiles if required.

-Prepare an agreed-upon list of Peoplesoft objects that you need to remigrate after the refresh is complete. Use below SQL to identify project differences in TST and PRD.    

             SELECT PROJECTNAME, COUNT(OBJECTTYPE)

             FROM PSPROJECTITEM

             GROUP BY PROJECTNAME;

-Ensure the last day export backup of TST is complete.

-Comment cronjobs on TST.

-Report Node backup- EXPORT PS_CDM_DIST_NODE

-Backup Web Profile -

-Export any security that you need to preserve from the target database to a flat file. Data Mover is a nice tool to use for this since you can qualify each table with a list of operator ID's to export. The tables you should consider exporting for specific users are PSOPRDEFN, PSOPRALIAS, PSROLEUSER, PSUSERATTR, PSUSEREMAIL, PSUSERPRSNLOPTN, PS_ROLEXLATOPR and PS_RTE_CNTL_RUSER. There is a delivered script - securityexport.dms which can be used for this task.

-Notify users about refresh and the downtime.

 

 

REFRESH Database -

 

-Stop the target application environment, including application servers and process schedulers. Stopping the web server is optional since it doesn't connect directly to the database. Be sure to clear cache.

-Turn off archive logging in TST

             SQL> shutdown immediate;

             SQL> startup mount;

             SQL> ALTER DATABASE NOARCHIVELOG;

             SQL> ALTER DATABASE OPEN;

-Remove archive logs (if any)Drop SCHEMA objects in TST and ensure that all objects are dropped.

-Import into the target database recent backup of the Production database.

-Set DBNAME in PSDBOWNER back to the target database name.

-Set GUID to ' ' in the PSOPTIONS table. This will cause PeopleSoft to generate a new GUID so that change assistant can track it separately from the source database.

-Using datamover bootstrap mode, reset the ACCESSID and its password, encrypt it. Change Access Password (SYSADM)-
    CHANGE_ACCESS_PASSWORD SYSADM TSTPSWD;


-In data mover, change the user passwords that are configured in your application server, process scheduler, and integration broker configurations back to the pre-refresh values:
UPDATE PSOPRDEFN SET OPERPSWD = 'devpswd', ENCRYPTED = 0 WHERE OPRID = ('PSAPPS');
ENCRYPT_PASSWORD PSAPPS;
UPDATE PSOPRDEFN SET OPERPSWD = 'devpswd', ENCRYPTED = 0 WHERE OPRID = ('PTWEBSERVER');
ENCRYPT_PASSWORD PTWEBSERVER;


-Verify grant to PEOPLE account- 

       SELECT FROM DBA_TAB_PRIVS WHERE GRANTEE=’PEOPLE’;  /* This should return SELECT privileges on three security records.

-Test 2-tier connectivity

Post-refresh activities -

-Delete the data from the reporting tables, process scheduler tables and application messaging tables since this data isn't relevant in the target database.                               

1) Run prcsclr.dms - this will cleanup process scheduler tables. (PSPRCSRQST, PSPRCSQUE, PSPRCSPARMS, PS_MESSAGE_LOG etc.)                                                  

2)Run rptclr.dms - this will cleanup Report Manager tables. (PS_CDM_LIST, PS_CDM_AUTH etc.)

3) Cleanup messaging tables, run appmsgpurgeall.dms 

-Truncate PSACCESSLOG and history tables.

-Import the security that you exported - run securityimport.dms. Important PS objects/tables that are imported using securityimport are-

o    Users - PSOPRDEFN, PSUSEREMAIL , PSUSERATTR , PSOPRALIAS , PSROLEUSE,

o    Roles - PSROLEDEFN, PSROLECLASS,

o    Access Profiles - PSACCESSPRFL, 

-Import Permission Lists, Definition Security records, and Personalization etc.

-Truncate PSACCESSLOG and history tables.

-Delete application server cache and restart the domain.

-Delete webserver cache and restart the webserver domain. Clear application server/web server and process scheduler cache. Start the target application environment.

-Test PIA signon.

-Manual online configuration- Navigate to PeopleTools > Utilities > Options and update Environment Long/Short name, TimeZone, F1 help URL and help URL on web profile configuration. Update the Report Node configuration and verify your Process Scheduler Servers are using the correct configuration.

-Clear Process Scheduler cache, import report node (PS_CDM_DIST_NODE). Reconfigure process scheduler, boot and test run a report, verify the report runs to completion, posts, and allows you to view the report without having to sign in a second time.

-Verify performance indexes, recreate missing one.

-Check for invalid objects, recreate if any.

-Run data scrambling scripts. Reset everyone's e-mail address to a pre-defined value so that workflow messages from the Test environment don't get sent to real users.  e.g. SSN#, telephone# etc., emails in PSUSEREMAIL, PSOPRDEFN, PS_ROLEXLATOPR etc.

-Re-enable archive logging-

        SQL> startup mount;

          SQL> ALTER DATABASE ARCHIVELOG;

          SQL> ALTER DATABASE OPEN;

          SQL> ALTER SYSTEM SWITCH LOGFILE;

-Run hot RMAN backup for refreshed environment. Run RMAN backup for archive logs.

Copy all of the batch objects (SQR's, Crystal Reports, COBOL programs, etc.) on both UNIX and Windows servers from your source to your target environment to keep the environment in synch.

-Re-migrate projects/SQRs.

-Notify users that refresh is complete and environment is available again.

* In a DB2 database, run stats/re-orgs on tables after refresh and prior to running data masking/scrambling scripts otherwise you may experience slower performance of these scripts.

 

Linux/ HP-UX/Solaris

Linux directory structure - tree like structure, with directories like-
etc- contains system configuration information,
tmp - contains temporary files,
user - contains user programs and utilities, includes subdirectories as /usr/bin, /usr/sbin and /usr/local/bin

/usr/bin - contains utilities to be used by end users. Some of these are -
awk - a pattern searching and processing language,
df - displays available disk space,
ftp - File Transfer Protocol tool,
gzip/zip - used to compress and decompress files,
ssh - secure shell processor,
sudo - used to run a program as root,
tail - reads from end of a file,
top - displays top CPU processes,
who - displays a list of users on the system.

/usr/sbin -
useradd - used to add new user accounts to the system,
groupadd - used to add groups to system.
crond - this daemon runs regularly scheduled jobs submitted by cron which might be user or system programs that run every hour, every day, every month etc.
sshd - this is the secure shell server, SSH clients connect through this daemon.

Network utilities - arp - Address Resolution Protocol.
ifconfig - used to configure and manage network adaptors.


Linux Web Server utilities - it is an open source HTTP web server that runs on many platforms including Linux/UNIX and Windows. It contains many modules e.g. a PERL module, an authentication module and a proxy module. In order to web users to access the Oracle db, a CGI or Common Gateway Interface is used.

Processes and Threads- Within the OS, many programs or processes are simultaneously running on the system. Each process has its own virtual memory space and run independently of other processes. Threads are concurrent threads of instructions which run within a single process.

Logical Volume Manager (LVM) and ASM (Automatic Storage Management) - LVM manages storage by combining physical disk blocks from physical disks into a logical disk volume. Oracle's ASM is an LVM which has been developed specifically for Oracle and simplifies Oracle storage management and optimize Oracle data storage.



Linux Commands/programs-
Commands are also referred as programs since whenever you run a command, it is the corresponding program code, written for the command that is being executed.

sar ( System Activity Reporter) - useful for monitoring the performance of system in general. There are multiple options with sar depending upon what you want to monitor. Using sar, you monitor CPU utilization, memory and network interfaces.

sar with no parameters displays the CPU utilization for last 24 hours. $sar -A displays a lot of system information for last 24 hours. $sar -n DEV displays information about network interfaces. This is a repeating command that will run at intervals and duration as specified e.g. $sar 10 100

vmstat - provides information about processes, memory, paging etc. Use this command to determine if there is free memory. This command can be run in a loop.
$vmstat 10 100 - specifies that the vmstat is run every 10 seconds for 100 times.

top- continually displays the top running processes and overall view of the activity on the system. $top -c displays the command line rather than just the command that is being run.

iostat - displays information about the I/O subsystem e.g. reads and writes per second, blocks written per second, wait time and response time. $iostat -k displays detailed I/O information. $iostat -x 10 100 runs 100 times at every 10 seconds. The important data points in output are - 1) await - average wait time, 2) svctm - the service time, 3) r/s - the number of reads per second and 4) w/s - number of writes per second.
the

ps - command provides process statistics, information on the processes that are currently running. There are a number of additional parameters that provide more detailed information on currently running processes. The second column of output from ($ps -elf) shows the process state code.

Different state codes are -
R, Runnable - process currently running on a CPU,
S, Sleeping - process is waiting for something to happen,
D, uninterrupted sleep usually indicates that the process is waiting an I/O.
X, Dead,
T, traced or stopped.

df - to find the size of file system. df -h displays the information in human readable format. K, M, G for kilobytes, Megabytes and Gigabytes. -h flag can be used with many file system commands e.g. df, du, ls etc.

sed (the stream editor) -  special editor for modifying files automatically.
s option with sed for substitution - the most useful command that changes all occurrences of a regular expression into a new value.
$sed s/d1sdt1d1/p1sdt1d1/ old > new   =>  the command changes d1sdt1d1 in old file to p1sdt1d1 in the new file.


Mostly used Linux commands- 

ls         Lists all files and directories in the present working directory
ls -R     Lists files in sub-directories as well
ls –a     Lists hidden files as well Lists hidden files as well
ls –la  Lists files and directories with detailed information like permissions , size , owner etc
cat > <filename> Creates a new file
cat <ilename> isplays the file content
cat file file2 > file3 Joins two files (file1, file2) and stores the output in a new file (file3)
mv  file “new file path” Moves the files to the new location
mv filename new_file_name  renames the file to a new filename
sudo Allows regular users to run programs with the security privileges of the superuser or root
rm Deletes a file
history Gives a  list of all past commands typed in the current terminal session
clear Clears the terminal
mkdir Creates a new directory in the present working directory
rmdir Deletes a directory
mv Renames a directory

tr copies the input to produce output with translation/substitution.
find used to locate files on UNIX system.
cut - cuts out selected fields of each line.


File Permissions in Linux -For effective security , Linux divides authorization into 2 levels
1) Ownership
2) Permission


Ownership in Linux files  - Every file and directory in your Unix/Linux system is assigned 3 types -
User - A user is the owner of the file. By default, the person who created a file becomes its owner.
Group - A user- group can contain multiple users. All users belonging to a group will have the same access permissions to the file. Instead of manually assigning permissions to each user, you could add all users to a group, and assign group permission to file such that only this group members and no one else can read or modify the files.
Other - Any other user; this person has neither created the file nor he belongs to a user-group who could own the file. When you set the permission for others, it is also referred as set permissions for world.

Permissions -
Every file and directory in your UNIX/Linux system has following 3 permissions defined for all the 3 owners discussed above
Read : This permission gives you the authority to open and read a file. Read permission on a directory gives you the ability to lists it’s content.
Write: The right permission gives you the authority to modify the contents of a file. The write permission on a directory gives you the authority to add , remove and rename files stored in the directory.  Consider a scenario where you have write permission on the file but do not have write permission on the directory where the file is stored. You will be able to modify the file contents. But you will not be able rename, move or remove the file from the directory.
Execute : In windows an executable program usually has an extension “.exe” and which you can easily run. In Unix/Linux , you cannot run a program unless the execute permission is set. If the execute permission is not set, you might still be able to see/modify the program code(provided read & write permissions are set), but not actually run it.

r = read permission
w = write permission
x = execute permission
- = no permission
chmod command which stands for ‘change mode’. Using the command, we can set permissions (read, write, execute) on a file/directory for the owner, group and the world.

There are 2 ways to use the command -

Absolute mode
Symbolic mode

1) Absolute(Numeric) Mode - In this mode file permissions are not represented as characters but a three digit octal number.


Number - Permission Type - Symbol
0 No Permission ---
1 Execute --x
2 Write -w-
3 Execute + Write -wx
4 Read r--
5 Read + Execute r-x
6 Read +Write rw-
7 Read + Write +Execute rwx


e.g. $chmod 770 filename - sets all permissions on filename for owner and group leaving no permissions for others.

2) Symbolic Mode -In the symbolic mode you can modify permissions of a specific owner. It makes use of mathematical symbols to modify the file permissions

+ Adds a permission to a file or directory
- Removes the permission
= Sets the permission and overrides the permissions set earlier.

The various owners are represented as -

e.g. $chmod o=rwx filename  - sets all the read+write+execute permissions on file for others.


u - user/owner
g - group
o - other
a - all


Changing Ownership and Group - For changing the ownership of a file/directory , you can use the following command:

$chown user filename
$chown user:group filename - In case you want to change the user as well as group for a file or directory use the command.


Change group - The file /etc/group contains all the groups defined in the system. The command “groups” lists all the groups you are a member of.
$groups
To change the group of a file, use

$chgrp newgroupname file_name


You cannot have 2 groups owning the same file. Though one group can be a sub-group of another which can own a file. You can use the command newgrp to work as a member a group other than your default group.

$newgrp oracle

Pipes in Linux -The symbol ‘|’ denotes a pipe. Pipes ‘|’ help combine 2 or more commands. If you want to use two or more commands at the same time and run them consecutively, you can use pipes.

$cat filename | less
$cat Filename | pg
$cat Filename | more


* less, pg and more commands are used for dividing a long file into readable bits.

The ‘grep’ command - the grep command scans the file for the desired information and present the result in a format you want. The grep command can be used to find strings and values in a text document. Following options can be used with this command.
-v Shows all the lines that do not match the searched string
-c Displays only the count of matching lines
-n Shows the matching line and its number
-i Match both (upper and lower) case
-l Shows just the name of the file with the string


$cat sample | grep -i a


The ‘sort’ command - This command helps in sorting out the contents of a file alphabetically. The options with this command are -

-r Reverses  sorting
-n Sorts numerically
-f Case insensitive sorting
$sort -r file

A filter in a pipe is an output of one command which serves as input to the next. When you pipe 2 commands, the “filtered ” output of the first command is given to the next.


Regular expressions - Regular expressions are a set of characters used to check patterns in strings. They are also called ‘regexp’ and ‘regex’. Some popular regular expressions are -
. replaces any character
^ matches start of string
$ matches end of string

$cat filename | grep ^<character> - only lines in a file that starts with the <character> are filtered in output.

Redirection - The standard input (stdin) device is the keyboard.
The standard output (stdout) device is the screen.


Output Redirection - The ‘>’ symbol is used for output (STDOUT) redirection. Example:
$ls –al > listings - the output of command ls –al is re-directed to file “listings” instead of your screen.

“>>” appends output to an existing file.
* You can re-direct standard output, to not just files, but also devices!

Input redirection - The ‘<’ symbol is used for input(STDIN) redirection. e.g.

$Mail –s “Subject” to-address < Filename

$Mail –s “News Today” abc@gmail.coms < Newsflash

Example - The mail program in Linux can help you send e-mails from the Terminal. 
You can type the contents of the email using the standard device keyboard. But if you want to attach a File to email you can use the input re-direction operator in above format.


“>&” which writes the output from one file to the input of another file.

Error Redirection - Whenever you execute a program/command at the terminal, 3 files are always open, viz., standard input, standard output, standard error.

By default, error stream is displayed on the screen. Error redirection is routing the errors to a file other than the screen. Error re-direction is one of the very popular features of Unix/Linux.

$myprogram 2>errorsfile -Here Using “2>” we re-direct the error output to a file named “errorfile”, thus, program output is not cluttered with errors.
$find . -name 'my*' 2>error.log  -Using the “find” command we are searching the “.” current directory for a file with “name” starting with “my”
$ls Documents ABC> dirlist 2>&1 - lists contents of the two directories - Document and ABC. Standard output is redirected to the file dirlist and error output(2) is written (by >&) to input of 1(standard output i.e. screen)


File Descriptors - In Linux/Unix everything is a file. Regular file, Directories and even Devices are files. Every File has an associated number called File Descriptor (FD).

FD0 - Standard Input STDIN
FD1 - Standard Output STDOUT
FD2 - Standard Error STDERR

Variables - unique names  given to memory locations/addresses in system RAM. There are two types of variables-
1) System variables - these are Linux created and maintained variables defined in CAPITALs
2) User Defined variables - created and maintained by user usually in lower case.

Environment variables - Dynamic values which affect the processes or programs on a computer. These can be created, edited, saved and deleted. e..g. $LANG environment variable stores the value of the language that the user understands.

Some common environment variables -
PATH - This variable contains a colon(:) separated list of directories in which your system looks for executable files. When you enter a command on terminal, the shell looks for the command in  different directories mentioned in the $PATH variable.
USER - The username.
HOME - Default path to the user’s home directory.
UID - User’s unique ID.
SHELL - Shell being used by the user.

$echo $VARIABLE - display value of a variable.
$VARIABLE_NAME= variable_value - Creates new variable.
$unset VARIABLE_NAME - to remove a Variable from the system. This would remove the $Variable and its value permanently.
$export Variable=value - To set value of an environment variable.


Monday

Application Hosting

Application hosting is a application distribution system where the applications e.g. Peoplesoft HCM, CRM etc. are maintained and managed by a Application Service Provider (ASP) and is made available to client over internet. Hosting provider is responsible for support, maintenance, installations, upgrades and security on 24x7 basis.

According to Gartner, Inc. Business process as a service (BPaaS) represent the largest segment(77%), Infrastructure as a service (IaaS) is the fastest-growing segment of the public cloud and Software as a service (SaaS) is the next-largest segment. According to a recent market report, 40% Of CRM Systems sold are SaaS-Based.


Considerations for application hosting-


  • Storage requirements
  • Environments to be hosted
  • Backup/restore services and DR
  • SLA monitoring and reporting
  • Root access/Security
  • Operational tasks
  • Testing
  • Licensing
  • Help Desk and System Support Model

Storage requirements: Use a just-in-time-inventory approach to defer costs until they are essential. How much storage do you need in the next 3-5 years? Break this into both SAN and NAS. How much is needed now? How much additional space is needed each year? Will you archive data over time and pull it out of the main storage area? How much and when?
How many environments to be hosted: Each environment has different requirements in terms of support, availability, operational tasks and access. E.g. Staging, training, testing, production and development.
Backup/restore services and DR: Define what is expected to be backed up, how often and when. What happens when there is a disaster and the data center is completely unusable? Do you have a continuity of operations plan?
SLA monitoring and reporting: Identify and define SLAs, Availability and core hours during which availability is required. What all to be monitored? What is the interval for monitoring and the threshold for those items being monitored?
Root access/Security: Who needs root access? Typically, system root access is not given to anyone outside the data center staff, but full application administrative access may be granted to application admin staff.
Operational tasks: Installation, configuration, steady state operations, tuning and database support tasks should be defined. Develop a Standard Operating Procedures Manual.
Testing: Define what types of testing is to be performed and who is to perform them before go live.
Licensing: Define who is responsible for acquiring software licenses and maintenance agreements, as well as who is the party responsible for paying for these licenses. 
Help Desk and System Support: How are the issues escalated, and which parties to be notified in the event of an issue/outage? Develop an escalation and notification matrix update process and review cycle.






Tuesday

Payroll for North America

Payroll Business Processes

The NAP is peoplesoft enterprise application to calculate earnings, taxes, deductions, maintain balances and report payroll data.

While designing the payroll system, you group your workforce and define when you want to pay them. Earnings, deductions, taxes and processes you define that best suite your business requirements. The system enables you to calculate gross-to-net or net-to-gross pay, leave accruals and direct deposits.

Setup Core Payroll Tables



Setup Employee Pay Tables


Process the Payroll


Post to General Ledger


Pay Taxes


Third Parties Payments


Reporting

Followers