If you use a VMFS3 datastore, you must upgrade it to VMFS5.

When upgrading the datastore, use the following option: 
-T|--upgradevmfs /vmfs/volumes/UUID
The upgrade is a one-way process. After you have converted a VMFS3 datastore to VMFS5, you cannot revert it back.

All hosts accessing the datastore must support VMFS5.