WordPress! Good old friend. It’s time for us to go on our own ways.
I’m moving to my own jekyll/disqus based blog at http://amin.bitbucket.org/
I won’t need hours to fix Java/Perl syntax and planning to write heaps on SmartOS/Telstra/DevOps/Java.
After I upgraded from Lion to Mountain Lion the MAC address of VMWare Fusion adaptor had changed for no good reason. That’s why SmartOS (illumos) was failing to plumb the network and dladm complains about unknown status.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| root$ dladm show-phys | |
| LINK MEDIA STATE SPEED DUPLEX DEVICE | |
| e1000g0 Ethernet unknown 1000 half e1000g0 |
To fix this check the newly generated MAC and update the /usbkey/config accordingly.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| grep ^admin_nic /usbkey/config | |
| admin_nic=0:c:29:fb:7f:64 |
Check the MAC in OS after reboot. Now interfaces should come up automatically at boot time.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| [root@00-0c-29-fb-7f-64 ~]# dladm show-phys -m | |
| LINK SLOT ADDRESS INUSE CLIENT | |
| e1000g0 primary 0:c:29:fb:7f:64 yes e1000g0 |
There seems to be a reasonable security feature in Solaris that limits usage of diagnostics tools based on privileges(5) of source and target PIDs. Saying that, tools such as dtrace(1M) and pstack(1) should have equal or more privileges(5) than the target PID they want to observe. Otherwise the process owner can use the target PID to run instrumented instructions with higher privileges which is obviously a security hole.
But this fair statement can cause some headache especially when processes start from sources other than a shell such as SMF.
Let’s examine this scenario. What prevents the owner of this process to look inside even after setting all dtrace permissions to zone and user.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ ppriv $$ | |
| 29763: -bash | |
| flags = <none> | |
| E: basic,dtrace_proc,dtrace_user,proc_owner | |
| I: basic,dtrace_proc,dtrace_user,proc_owner | |
| P: basic,dtrace_proc,dtrace_user,proc_owner | |
| L: basic,contract_event,contract_observer,dtrace_proc,dtrace_user,file_chown,file_chown_self, | |
| file_dac_execute,file_dac_read,file_dac_search,file_dac_write,file_owner,file_setid,ipc_dac_read, | |
| ipc_dac_write,ipc_owner,net_bindmlp,net_icmpaccess,net_mac_aware,net_privaddr,net_rawaccess, | |
| proc_audit,proc_chroot,proc_lock_memory,proc_owner,proc_setid,proc_taskid,sys_acct,sys_admin, | |
| sys_audit,sys_mount,sys_nfs,sys_resource | |
| $ /usr/ucb/ps auxwww | grep java | fgrep "XX:+ExtendedDTraceProbes" | awk '{print $1,$2}' | |
| weblogic 1745 | |
| weblogic 27317 | |
| $ pfexec dtrace -n 'hotspot$target:::object-alloc{ @ = quantize(arg1) }' -p 1745 | |
| dtrace: failed to grab pid 1745: permission denied | |
| $ pstack 1745 | |
| pstack: cannot examine 1745: permission denied |
What’s up? The reason is hiding somewhere in the SMF service manifest. Let’s have a look:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <exec_method name='start' type='method' exec=…> | |
| <method_context> | |
| <method_credential user='weblogic' privileges='basic,sys_resource,…'/> | |
| </method_context> | |
| </exec_method> |
Looking at the PID of the service we notice it has extra sys_resource privilege assigned via SMF that we don’t have in our bash PID ($$).
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ ppriv 1745 | |
| flags = <none> | |
| E: basic,sys_resource,… | |
| I: basic,sys_resource,… | |
| P: basic,sys_resource,… | |
| ….. |
That’s preventing bash PID to access SMF started service PID although they are owned by the same user. So what is this extra privilege?
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ ppriv -lv sys_resource | |
| sys_resource | |
| Allows a process to modify the resource limits specified | |
| by setrlimit(2) and setrctl(2) without restriction. | |
| Allows a process to exceed the per-user maximum number of processes. | |
| Allows a process to extend or create files on a filesystem that | |
| has less than minfree space in reserve. | |
That’s a necessary for this service. So have to add the missing to the user:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| usermod -K defaultpriv=basic,sys_resource,… weblogic |
Next time bash has sys_resource privilege and executes dtrace/pstack against SMF service successfully:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ ppriv $$ | |
| 8630: -bash | |
| flags = <none> | |
| E: basic,dtrace_proc,dtrace_user,proc_owner,sys_resource | |
| I: basic,dtrace_proc,dtrace_user,proc_owner,sys_resource | |
| P: basic,dtrace_proc,dtrace_user,proc_owner,sys_resource | |
| L: basic,contract_event,contract_observer,dtrace_proc,dtrace_user,file_chown,file_chown_self,file_dac_execute,file_dac_read,file_dac_search,file_dac_write,file_owner,file_setid,ipc_dac_read,ipc_dac_write,ipc_owner,net_bindmlp,net_icmpaccess,net_mac_aware,net_privaddr,net_rawaccess,proc_audit,proc_chroot,proc_lock_memory,proc_owner,proc_setid,proc_taskid,sys_acct,sys_admin,sys_audit,sys_mount,sys_nfs,sys_resource | |
| $ pfexec dtrace -n 'hotspot$target:::object-alloc{ @ = quantize(arg1) }' -p 1745 | |
| dtrace: description 'hotspot$target:::object-alloc' matched 1 probe | |
| ^C | |
| value ————- Distribution ————- count | |
| 536870912 | 0 | |
| 1073741824 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ 8945 | |
| 2147483648 | 0 |
When importing a virtual appliance from a VT-x machine into a non VT-x one, you may end up with this error:
VT-x is not available: verr_vmx_no_vmx
Well the key to fix this is to disable VT-x (i.e. hardware virtualization instructions) but the problem is that the new host machine’s virtualbox settings window is all disabled:
So how can I disable VT-x then? Then solution I found it to open the appliance vbox XML file and manually disable HardwareVirtualEx.
vim ~/VirtualBox\ VMs/windows/windows.vbox
Then search for HardwareVirtEx and set all enabled items to false.
1. Grab these two files: FSWpart.tar.gz and FSWfsmisc.tar.gz
2. Rename files:
# mv fswpart-tar-gz-remove.png FSWpart.tar.gz # mv fswfsmisc-tar-gz-remove.png FSWfsmisc.tar.gz
3. Install the packages.
# tar -zxvf FSWpart.tar.gz # tar -zxvf FSWfsmisc.tar.gz # pkgadd -d . FSWpart # pkgadd -d . FSWfsmisc
The following files are already installed on the system and are being used by another package: * /etc/gnome-vfs-2.0 <attribute change only> * /etc/gnome-vfs-2.0/modules <attribute change only> * /usr/lib/gnome-vfs-2.0 <attribute change only> * /usr/lib/gnome-vfs-2.0/modules <attribute change only> * /usr/sbin/mkntfs * /usr/sbin/ntfsclone * /usr/sbin/ntfscp * /usr/sbin/ntfslabel * /usr/sbin/ntfsresize * /usr/sbin/ntfsundelete * - conflict with a file which does not belong to any package. Do you want to install these conflicting files [y,n,?,q] y
4. Find the target partition
# iostat -En
5. Mount partitions in the devices into folders you like:
# mkdir /mnt/d # mount -F ntfs /dev/dsk/c5t0d0p3 /mnt/d
6. Add mount targets to /etc/vfstab
/dev/dsk/c5t0d0p2 - /mnt/c ntfs - yes - /dev/dsk/c5t0d0p3 - /mnt/d ntfs - yes -
If you’re connected to internet and still some CLI tools such as curl, python, easy_install and pip fail with errors such as:
curl: (7) Failed to connect to IP: Host is down
error: [Errno 64] Host is down
This is while host is ping from command line and browser connects too. First check you don’t have any proxy settings. (echo $http_proxy)
If you don’t have any proxy settings, and browser accesses the site, well then it might be because of some strange rules in LittleSnitch in the case you have one running. Try to stop LittleSnitch network monitor and give it a go.
Just until recently I believed having multiple desktops could work same as two monitors but I proved myself wrong last week.
For most of us, daily work is a combination of routine stuff (emails, maintenance, phone-calls, meetings) and development. The problem is when this flood of routine tasks take most all of our time and attention.
By having two monitors one can easily partition this two stream and keep and eye on both. Here is how my desk looks like these days.
In one monitor I have my outlook, communicator, GTD and in the second one all dev tools (iTerm, IntelliJ, emacs, sql*plus, etc) in hand.
I also noticed how good x86 is for development rather than SPARC. Having so many cores on one SPARC is great for scalable production deployment but not necessarily suitable enough for development. I decided to run Solaris in my PC and connect monitor/keyboard to this darling little beast; MacBook Air.
1. Grab RCU for Linux and extract. Go to rcuHome/bin folder.
2. Disable platform check in ./rcu script:
3. Create a Link from JDK to RCU:
4. Change JRE Path in ./rcu script:
Solaris has so many features and the more you use it, the more you find. Before moving to Telstra my (painful) experience with Solaris dated back almost 10 years ago with Solaris 7 and 8. Mostly because at that days it was among the limited options of a reliable platform for running mission critical Oracle RDBMS instances (I guess it is still).
But then starting again with Solaris 10 in Telstra I found the huge progress it had made. I always thank Oracle for saving Solaris and do hope that the continuation the good work although unfortunately we hear news that many brains behind this master-piece have left the Oracle.
IMHO here’s the list some of the most beloved features of Solaris 10. These are also the feature that don’t have a proper equivalent in other competitors.
- Containers: real virtualization not buzzword
- ZFS: nothing comes even close to this File system. What on earth they had in mind.
- DTrace: keep tuning/monitoring live applications without major overhead
- SMF: far better that init.d especially with contracts
Some may complain the Linux has OpenVZ/LXC or Mac OS X has ZFS and DTrace. but the question remains: are they all production ready? Even if they are, no doubt they got the idea from Solaris and tried to reimplement.
Enrollment Issue
I spent couple of hours working on a problem that prevents some of our WLSM instances to fire up correctly. The issue originated from a NullPoniterException in PDClient. PDClient is required for WLSM instances running in controlled (pull/push) mode. NPE was something like this:
| <Mar 28, 2012 3:26:10 PM GMT+10:00> <Error> <HTTP> <BEA-101216> <Servlet: “PDClientServiceServlethttp” failed to preload on startup in Web application : “pd-client.war”.javax.xml.ws.WebServiceException: java.lang.NullPointerExceptionat weblogic.wsee.jaxws.WLSInstanceResolver.getSingleton(WLSInstanceResolver.java:36) at weblogic.wsee.jaxws.WLSInstanceResolver.start(WLSInstanceResolver.java:55) Caused By: java.lang.NullPointerException at oracle.security.jps.soap.pd.client.PDClient.<init>(PDClient.java:46) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) Truncated. see log file for complete stacktrace |
In short, although not much vivid but if you also encounter this error it’s possibly due to immature enrolment of the SM instance.
For Server based SMs (e.g. Weblogic, WebSphere, and JBoss), OESSM also creates a config inside server’s smconfig folder besides the original one under $OES_CLIENT_HOME/oes_sm_instances.
For WLSM, the configuration is under WLS_DOMAIN/config/oeswlssmconfig folder. The JPS configuration in this folder is almost independent (see jps-config.xml ) but has a small link back to original config regarding enrolment wallet. See:
| <serviceInstance location=”/oracle/Middleware/oes_client/oes_sm_instances/<SM name>/config/enroll” provider=”credstoressp” name=”credstore.enroll”/> |
If the enrolment of the SM has failed, then the cwallet.sso will be still there but in an incomplete mode which prevents the correct start up of the PDClient.
How to check if Wallet is correct?
OES server comes with ORAPKI a handy tool to inspect cwallet files. You can find it under $ORACLE_MIDDLEWARE_HOME/oracle_common/bin/
Here is how to display the contents of a binary wallet file:
| /oracle/Middleware/oracle_common/bin/orapki wallet display -wallet <cwallet.sso> |
For a correctly enrolled file, the contents include two user certificate entries for Oracle Secret Store.
| -bash-3.00$ /oracle/Middleware/oracle_common/bin/orapki wallet display -wallet ../../<SM Name>/config/enroll/cwallet.sso Oracle PKI Tool : Version 11.1.1.5.0 Copyright (c) 2004, 2011, Oracle and/or its affiliates. All rights reserved. Requested Certificates: User Certificates: Oracle Secret Store entries: OES_SYMMETRIC_KEY_MAP@#3#@OES_IV_PARAMETER_alias OES_SYMMETRIC_KEY_MAP@#3#@OES_SYMMETRIC_KEY_alias Trusted Certificates: Subject: OU=Class 2 Public Primary Certification Authority,O=VeriSign\, Inc.,C=US Subject: OU=Class 3 Public Primary Certification Authority,O=VeriSign\, Inc.,C=US Subject: CN=GTE CyberTrust Global Root,OU=GTE CyberTrust Solutions\, Inc.,O=GTE Corporation,C=US Subject: OU=Class 1 Public Primary Certification Authority,O=VeriSign\, Inc.,C=US |
While an immature one lacks these two:
| -bash-3.00$ /oracle/Middleware/oracle_common/bin/orapki wallet display -wallet ../../<SM name>/config/enroll/cwallet.ssoOracle PKI Tool : Version 11.1.1.5.0 Copyright (c) 2004, 2011, Oracle and/or its affiliates. All rights reserved. Requested Certificates: User Certificates: Trusted Certificates: Subject: OU=Class 2 Public Primary Certification Authority,O=VeriSign\, Inc.,C=US Subject: OU=Class 3 Public Primary Certification Authority,O=VeriSign\, Inc.,C=US Subject: CN=GTE CyberTrust Global Root,OU=GTE CyberTrust Solutions\, Inc.,O=GTE Corporation,C=US Subject: OU=Class 1 Public Primary Certification Authority,O=VeriSign\, Inc.,C=US |
How to (Re) Enroll an SM?
There are couple of scripts generated by OESSM tool to initialize key and perform enrolment (under bin folder of SM instance) but there is my approach:
| # prepare WLST env . /oracle/Middleware/wlserver_10.3/server/bin/setWLSEnv.sh # prepare OES env |
PS1: InitEnrolment is the only step required for controlled-pull SMs, because they directly talk to Policy Store DB. For controlled-push SMs, need an extra step of DoEnrolment (see config.sh)
PS2: Try to use JDK 1.6.26+
Amin Abbaspour's Blog 
