[python]__dict__ attribute of an object - values not accessible

[RedBullAddicted]

Hi,

Maybe someone can explain that weird behavior to me. I really don't get it I try to get some infos from a vcenter server about used datastores using pysphere.

First I simply connect, get the datastores and their properties and get the hosts that have that specific datastore mounted:

Code: (python) [Select]

>>> from pysphere import VIServer, VIProperty
>>> import inspect
>>> server = VIServer()
>>> server.connect("server", "user", "password")
>>> dsMorList = []
>>> for ds_mor, ds_name in server.get_datastores().items():
...   dsMorList.append(ds_mor)
...
>>> propsList = []
>>> for ds_mor in dsMorList:
...   propsList.append(VIProperty(server, ds_mor))
...
>>> hostList = []
>>> for props in propsList:
...   for host in props.host:
...     hostList.append(host)
...
>>>

so far nothing fancy and pretty simple. Now I need the hostname of the returned host object. I inspected the object and found out that it has a method called key which has a __dict__ attribute. Inside that dictionary is another dictionary called _values and the key for the hostname is "name".

Code: (python) [Select]

>>> hostList[0].key._values["name"]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/pysphere/vi_property.py", line 74, in __getattr__
    raise AttributeError("object has not attribute %s" % name)
AttributeError: object has not attribute _values
>>> hostList[0].key._values["name"]
'hostname01.domain.local'

As you can see I get an AttributeError on the first try and the expected result on the second try. If I do an inspect of the object first I can access the desired value directly afterwards:

Code: (python) [Select]

>>> inspect.getmembers(hostList[1].key)
[('__class__', <class 'pysphere.vi_property.VIProperty'>), ('__delattr__', <method-wrapper '__delattr__' of VIProperty object at 0x6d8c910>), ('__dict__', {'_server': <pysphere.vi_server.VIServer instance at 0x3d14050>, '_obj': 'host-1966', '_type': 'ManagedObjectReference', '_values': {'effectiveRole': <pysphere.resources.VimService_services_types.ArrayOfInt_Holder object at 0x1140ae90>, 'alarmActionsEnabled': True, 'configStatus': 'green', 'disabledMethod': <pysphere.resources.VimService_services_types.ArrayOfString_Holder object at 0x1140ac90>, 'hardware': <pysphere.resources.VimService_services_types.DynamicData_Holder object at 0x13ba6d10>, 'tag': <pysphere.resources.VimService_services_types.ArrayOfTag_Holder object at 0x14009610>, 'value': <pysphere.resources.VimService_services_types.ArrayOfCustomFieldValue_Holder object at 0x14009e50>, 'datastore': <pysphere.resources.VimService_services_types.ArrayOfManagedObjectReference_Holder object at 0x11405110>, 'triggeredAlarmState': <pysphere.resources.VimService_services_types.ArrayOfAlarmState_Holder object at 0x14009c10>, 'network': <pysphere.resources.VimService_services_types.ArrayOfManagedObjectReference_Holder object at 0x13bc8390>, 'configIssue': <pysphere.resources.VimService_services_types.ArrayOfEvent_Holder object at 0x13b9c6d0>, 'systemResources': <pysphere.resources.VimService_services_types.DynamicData_Holder object at 0x13d9a950>, 'recentTask': <pysphere.resources.VimService_services_types.ArrayOfManagedObjectReference_Holder object at 0x13bc8b50>, 'availableField': <pysphere.resources.VimService_services_types.ArrayOfCustomFieldDef_Holder object at 0x70dba10>, 'config': <pysphere.resources.VimService_services_types.DynamicData_Holder object at 0x113f28d0>, 'parent': 'domain-c1498', 'permission': <pysphere.resources.VimService_services_types.ArrayOfPermission_Holder object at 0x13bc88d0>, 'vm': <pysphere.resources.VimService_services_types.ArrayOfManagedObjectReference_Holder object at 0x1400d9d0>, 'datastoreBrowser': 'datastoreBrowser-host-1966', 'licensableResource': <pysphere.resources.VimService_services_types.DynamicData_Holder object at 0x13bc3850>, 'customValue': <pysphere.resources.VimService_services_types.ArrayOfCustomFieldValue_Holder object at 0x113ff590>, 'name': 'hostname02.domain.local', 'declaredAlarmState': <pysphere.resources.VimService_services_types.ArrayOfAlarmState_Holder object at 0x11405550>, 'summary': <pysphere.resources.VimService_services_types.DynamicData_Holder object at 0x13cd7a50>, 'capability': <pysphere.resources.VimService_services_types.DynamicData_Holder object at 0x85f7290>, 'configManager': <pysphere.resources.VimService_services_types.DynamicData_Holder object at 0x113fb0d0>, 'runtime': <pysphere.resources.VimService_services_types.DynamicData_Holder object at 0x13cb1850>, 'overallStatus': 'green'}, '_values_set': True}), ('__doc__', None), ('__format__', <built-in method __format__ of VIProperty object at 0x6d8c910>), ('__getattr__', <bound method VIProperty.__getattr__ of <pysphere.vi_property.VIProperty object at 0x6d8c910>>), ('__getattribute__', <method-wrapper '__getattribute__' of VIProperty object at 0x6d8c910>), ('__hash__', <method-wrapper '__hash__' of VIProperty object at 0x6d8c910>), ('__init__', <bound method VIProperty.__init__ of <pysphere.vi_property.VIProperty object at 0x6d8c910>>), ('__module__', 'pysphere.vi_property'), ('__new__', <built-in method __new__ of type object at 0x886600>), ('__reduce__', <built-in method __reduce__ of VIProperty object at 0x6d8c910>), ('__reduce_ex__', <built-in method __reduce_ex__ of VIProperty object at 0x6d8c910>), ('__repr__', <method-wrapper '__repr__' of VIProperty object at 0x6d8c910>), ('__setattr__', <method-wrapper '__setattr__' of VIProperty object at 0x6d8c910>), ('__sizeof__', <built-in method __sizeof__ of VIProperty object at 0x6d8c910>), ('__str__', <method-wrapper '__str__' of VIProperty object at 0x6d8c910>), ('__subclasshook__', <built-in method __subclasshook__ of type object at 0x1fc57f0>), ('__weakref__', None), ('_flush_cache', <bound method VIProperty._flush_cache of <pysphere.vi_property.VIProperty object at 0x6d8c910>>), ('_get_all', <bound method VIProperty._get_all of <pysphere.vi_property.VIProperty object at 0x6d8c910>>), ('_get_prop_value', <bound method VIProperty._get_prop_value of <pysphere.vi_property.VIProperty object at 0x6d8c910>>), ('_obj', 'host-1966'), ('_server', <pysphere.vi_server.VIServer instance at 0x3d14050>), ('_type', 'ManagedObjectReference'), ('_values_set', True)]
>>> hostList[1].key._values["name"]
'hostname02.domain.local'

Something else I wanted to add

Code: (python) [Select]

>>> hostList[5].key.__dict__
{'_server': <pysphere.vi_server.VIServer instance at 0x3d14050>, '_obj': 'host-2069', '_type': 'ManagedObjectReference', '_values_set': False}

>>> inspect.getmembers(hostList[5].key)
[('__class__', <class 'pysphere.vi_property.VIProperty'>), ('__delattr__', <method-wrapper '__delattr__' of VIProperty object at 0x70c20d0>), ('__dict__', {'_server': <pysphere.vi_server.VIServer instance at 0x3d14050>, '_obj': 'host-2069', '_type': 'ManagedObjectReference', '_values': {'effectiveRole': <pysphere.resources.VimService_services_types.ArrayOfInt_Holder object at 0x737c610>, 'alarmActionsEnabled': True, 'configStatus': 'green', 'disabledMethod': <pysphere.resources.VimService_services_types.ArrayOfString_Holder object at 0x737c410>, 'hardware': <pysphere.resources.VimService_services_types.DynamicData_Holder object at 0x7383490>, 'tag': <pysphere.resources.VimService_services_types.ArrayOfTag_Holder object at 0x11637110>, 'value': <pysphere.resources.VimService_services_types.ArrayOfCustomFieldValue_Holder object at 0x116378d0>, 'datastore': <pysphere.resources.VimService_services_types.ArrayOfManagedObjectReference_Holder object at 0xcfe8850>, 'triggeredAlarmState': <pysphere.resources.VimService_services_types.ArrayOfAlarmState_Holder object at 0x11637690>, 'network': <pysphere.resources.VimService_services_types.ArrayOfManagedObjectReference_Holder object at 0x78b3ad0>, 'configIssue': <pysphere.resources.VimService_services_types.ArrayOfEvent_Holder object at 0x7375e10>, 'systemResources': <pysphere.resources.VimService_services_types.DynamicData_Holder object at 0x79ad0d0>, 'recentTask': <pysphere.resources.VimService_services_types.ArrayOfManagedObjectReference_Holder object at 0x78b82d0>, 'availableField': <pysphere.resources.VimService_services_types.ArrayOfCustomFieldDef_Holder object at 0x16b781d0>, 'config': <pysphere.resources.VimService_services_types.DynamicData_Holder object at 0xcfd2a50>, 'parent': 'domain-c1498', 'permission': <pysphere.resources.VimService_services_types.ArrayOfPermission_Holder object at 0x78b8050>, 'vm': <pysphere.resources.VimService_services_types.ArrayOfManagedObjectReference_Holder object at 0x1163b450>, 'datastoreBrowser': 'datastoreBrowser-host-2069', 'licensableResource': <pysphere.resources.VimService_services_types.DynamicData_Holder object at 0x739af90>, 'customValue': <pysphere.resources.VimService_services_types.ArrayOfCustomFieldValue_Holder object at 0xcfe5cd0>, 'name': 'hostname05.domain.local', 'declaredAlarmState': <pysphere.resources.VimService_services_types.ArrayOfAlarmState_Holder object at 0xcfe8c90>, 'summary': <pysphere.resources.VimService_services_types.DynamicData_Holder object at 0x78e31d0>, 'capability': <pysphere.resources.VimService_services_types.DynamicData_Holder object at 0x9dec410>, 'configManager': <pysphere.resources.VimService_services_types.DynamicData_Holder object at 0xcfdf810>, 'runtime': <pysphere.resources.VimService_services_types.DynamicData_Holder object at 0x78b8f90>, 'overallStatus': 'green'}, '_values_set': True}), ('__doc__', None), ('__format__', <built-in method __format__ of VIProperty object at 0x70c20d0>), ('__getattr__', <bound method VIProperty.__getattr__ of <pysphere.vi_property.VIProperty object at 0x70c20d0>>), ('__getattribute__', <method-wrapper '__getattribute__' of VIProperty object at 0x70c20d0>), ('__hash__', <method-wrapper '__hash__' of VIProperty object at 0x70c20d0>), ('__init__', <bound method VIProperty.__init__ of <pysphere.vi_property.VIProperty object at 0x70c20d0>>), ('__module__', 'pysphere.vi_property'), ('__new__', <built-in method __new__ of type object at 0x886600>), ('__reduce__', <built-in method __reduce__ of VIProperty object at 0x70c20d0>), ('__reduce_ex__', <built-in method __reduce_ex__ of VIProperty object at 0x70c20d0>), ('__repr__', <method-wrapper '__repr__' of VIProperty object at 0x70c20d0>), ('__setattr__', <method-wrapper '__setattr__' of VIProperty object at 0x70c20d0>), ('__sizeof__', <built-in method __sizeof__ of VIProperty object at 0x70c20d0>), ('__str__', <method-wrapper '__str__' of VIProperty object at 0x70c20d0>), ('__subclasshook__', <built-in method __subclasshook__ of type object at 0x1fc57f0>), ('__weakref__', None), ('_flush_cache', <bound method VIProperty._flush_cache of <pysphere.vi_property.VIProperty object at 0x70c20d0>>), ('_get_all', <bound method VIProperty._get_all of <pysphere.vi_property.VIProperty object at 0x70c20d0>>), ('_get_prop_value', <bound method VIProperty._get_prop_value of <pysphere.vi_property.VIProperty object at 0x70c20d0>>), ('_obj', 'host-2069'), ('_server', <pysphere.vi_server.VIServer instance at 0x3d14050>), ('_type', 'ManagedObjectReference'), ('_values_set', True)]

>>> hostList[5].key.__dict__
{'_server': <pysphere.vi_server.VIServer instance at 0x3d14050>, '_obj': 'host-2069', '_type': 'ManagedObjectReference', '_values': {'effectiveRole': <pysphere.resources.VimService_services_types.ArrayOfInt_Holder object at 0x737c610>, 'alarmActionsEnabled': True, 'configStatus': 'green', 'disabledMethod': <pysphere.resources.VimService_services_types.ArrayOfString_Holder object at 0x737c410>, 'hardware': <pysphere.resources.VimService_services_types.DynamicData_Holder object at 0x7383490>, 'tag': <pysphere.resources.VimService_services_types.ArrayOfTag_Holder object at 0x11637110>, 'value': <pysphere.resources.VimService_services_types.ArrayOfCustomFieldValue_Holder object at 0x116378d0>, 'datastore': <pysphere.resources.VimService_services_types.ArrayOfManagedObjectReference_Holder object at 0xcfe8850>, 'triggeredAlarmState': <pysphere.resources.VimService_services_types.ArrayOfAlarmState_Holder object at 0x11637690>, 'network': <pysphere.resources.VimService_services_types.ArrayOfManagedObjectReference_Holder object at 0x78b3ad0>, 'configIssue': <pysphere.resources.VimService_services_types.ArrayOfEvent_Holder object at 0x7375e10>, 'systemResources': <pysphere.resources.VimService_services_types.DynamicData_Holder object at 0x79ad0d0>, 'recentTask': <pysphere.resources.VimService_services_types.ArrayOfManagedObjectReference_Holder object at 0x78b82d0>, 'availableField': <pysphere.resources.VimService_services_types.ArrayOfCustomFieldDef_Holder object at 0x16b781d0>, 'config': <pysphere.resources.VimService_services_types.DynamicData_Holder object at 0xcfd2a50>, 'parent': 'domain-c1498', 'permission': <pysphere.resources.VimService_services_types.ArrayOfPermission_Holder object at 0x78b8050>, 'vm': <pysphere.resources.VimService_services_types.ArrayOfManagedObjectReference_Holder object at 0x1163b450>, 'datastoreBrowser': 'datastoreBrowser-host-2069', 'licensableResource': <pysphere.resources.VimService_services_types.DynamicData_Holder object at 0x739af90>, 'customValue': <pysphere.resources.VimService_services_types.ArrayOfCustomFieldValue_Holder object at 0xcfe5cd0>, 'name': 'hostname05.domain.local', 'declaredAlarmState': <pysphere.resources.VimService_services_types.ArrayOfAlarmState_Holder object at 0xcfe8c90>, 'summary': <pysphere.resources.VimService_services_types.DynamicData_Holder object at 0x78e31d0>, 'capability': <pysphere.resources.VimService_services_types.DynamicData_Holder object at 0x9dec410>, 'configManager': <pysphere.resources.VimService_services_types.DynamicData_Holder object at 0xcfdf810>, 'runtime': <pysphere.resources.VimService_services_types.DynamicData_Holder object at 0x78b8f90>, 'overallStatus': 'green'}, '_values_set': True}


Why can't I access the needed data on the first go? I just don't get it. What am I doing wrong?

Thanks for your help

[HTH]

Well, i fired up python to check it out but i dont have a server to play with so instead i looked at the source, it seems when you run what you do you run this:

Code: [Select]

   def __getattr__(self, name):
        if not self._values_set:
            self._get_all()
 
        if not name in self._values:
            raise AttributeError("object has not attribute %s" % name)
 
        ret = self._get_prop_value(self._values[name])
        #cache the object
        setattr(self, name, ret)
        return ret
Id be willing to bet that for osme reason your hostname isnt inside the __values dictionary until those last two lines run once (ret =.... && setattr())

I cant say for sure but i think its worth a shot to change that code around and see if for some reason that __values dict isnt getting populated the way we think it should. I see nothing you are doing wrong and cant really see why to be entirely honest, but hey, its worth a shot right? (worst case just remove the  exception and run it twice )

[RedBullAddicted]

Thanks for your help HTH  I had a similar idea but I want to avoid to edit the library. I want the script to run on different machines and it should still work when pysphere gets update and such. So I did a pretty ugly

Code: (python) [Select]

try:
    hostList[0].key._values["name"]
except AttributeError:
    hostList[0].key._values["name"]

Even when its pretty bad it does work. The downside is that the script runs for more than 30 minutes to get the names of 462 host objects. Thats not ok. I think I just have to find another way to get the information I need.

[d4rkcat]

Even when its pretty bad it does work. The downside is that the script runs for more than 30 minutes to get the names of 462 host objects. Thats not ok. I think I just have to find another way to get the information I need.

As HTH already said, it's kind of hard to debug what you are trying to accomplish without access to a server, but from my limited understanding I would suggest threading the piece of code that needs to retrieve something from the web, and once all have been collected, inserting them into the dict.

[HTH]

As HTH already said, it's kind of hard to debug what you are trying to accomplish without access to a server, but from my limited understanding I would suggest threading the piece of code that needs to retrieve something from the web, and once all have been collected, inserting them into the dict.

I concur, if rba meets me on IRC in #coding sometimne tonight I;m sure we can thread it (since i presume most of the delay exists in the network) and cut that down to ~5 minutes easy enough.

[RedBullAddicted]

I already implemented threading and found out that our vmware ppl. limited the concurrent connections to the vcenter server to 10. But thats not a issue anymore. I searched around in the api docs and found a method to receive all host informations. The host object has a managed object identifier I can also find in the datastore object. So I just retrieve all host informations and compare the managed object identifier. The complete script with 10 threads needs round about 2 minutes to execute. Thats pretty ok for me now. Thanks for your help