Learn about the internals of a Python Object

Reading time: 25 minutes | Coding time: 5 minutes

In Object Oriented Python, it is useful to understand the internal things of an object or a class or a class instance. It can help in understanding the internal logic and in debugging.

In this article, you will learn about:

• where are variables and functions stored in Python?
• Internal data of objects and classes
• bound and unbound methods

Any object of a class has attributes and methods which are stored at particular memory address.

Consider a simple example of class and a class instance as follows:

class Class:
  ClassVar = 0

  def __init__(self):
    self.InstanceVar = 1

  def func(self):
    return None

c = Class()

Extract information from a class

There are two functions to extract information for a class:

• dir(...)

It is an inbuilt function in Python3, which returns a list of the attributes and methods of an object.

• vars(...)

Return the dict attribute for class, instance, or any object which has it.

Inside class

print(dir(Class))
print(vars(Class)) # or __dict__

Output:

['ClassVar', '__doc__', '__init__', '__module__', 'func']
{'__module__': '__main__', 'ClassVar': 0, 'func': <function func at 0x0000000002C5AEB8>, '__init__': <function __init__ at 0x0000000002C5AE48>, '__doc__': None}

Inside class instance

print(dir(c))
print(vars(c)) # or __dict__

Output:

['ClassVar', 'InstanceVar', '__doc__', '__init__', '__module__', 'func']
{'InstanceVar': 1}

Class variables vs instance variables

Now, suppose we change the value of ClassVar though Class object.

c1 = Class()
c2 = Class()
print(c1.ClassVar, c2.ClassVar)
Class.ClassVar = 3
print(c1.ClassVar, c2.ClassVar)

Output:

(0, 0)
(3, 3)

We can say that changing class variables through class affect all instances.

Now, let's change the value of ClassVar through Class instance object.

c1 = Class()
c2 = Class()
print(c1.ClassVar, c2.ClassVar)
c1.ClassVar = 3
print(c1.ClassVar, c2.ClassVar)

Output:

(0, 0)
(3, 0)

Python creates a new instance variable with the same name as a class variable and instance variables have precedence over class variables when searching for an attribute value, hence the output i.e. value changes only for c1.ClassVar.

Now, this is only valid for immutable types. Mutable class variables are not re-created. So, even if you change them through instance, they will be updated for all. Examples: list(), dict(), ...

Where are the variables stored in Python?

Python stores instance variables in a specialized dictionary variable dict which can be used for copying, debugging and many other use cases.

Where are the functions stored in Python?

As functions are not variables, they are found in dict of owner class object.

Overriding the attribute accessing methods

getattr

Now, after knowing how variables, functions are stored internally, i.e. in __dict__, let see how we can change the behaviour to access them.

Python class has __getattr__(self, name) overridable function. It can be used to update or provide custom logic to getting an attribute from an instance.

class Attributes(object):
  def __getattr__(self, name):
    return self.__dict__[name] # or some other 
a = Attributes()
print(a.key) # It will print a.__dict__[key] 

There is also __getattribute__ function. The difference is that getattr only gets called for whenever you request an attribute that hasn't already been defined before. While getattribute will get called always.

class SuperSecretClass(object):
  def __getattribute__(self, name):
    return None 
a = SuperSecretClass()
print(a.key) # Always None

setattr

This is same as __getattr__, instead is used to set the value of an attribute.
def __setattr__(self, name, value)

class Attributes(object):
  def __setattr__(self, name, value):
    if name == 'key':
      raise Exception('Not allowed!')
    self.__dict__[name] = value
a = Attributes()
a.key1 = None # Ok
a.key = None # Throws error 'Not allowed!'

delattr

This is also same as __getattr__, instead is used to remove the attribute.
def __delattr__(self, name)

class Attributes(object):
  def __delattr__(self, name):
    if name == 'key':
      raise Exception('Not allowed!')
    del self.__dict__[name]

a = Attributes()
a.key1 = None
a.key = None
del a.key1 # Ok
del a.key # Throws error 'Not allowed!'

Attribute lookup order in inheritance

• Data descriptors from class dictionary and its parents
• Instance dictionary
• Non-data descriptors from class dictionary and its parents

If an object defines both __get__ and __set__, it is a data descriptor.
If an object defines only __get__ it is a non-data descriptor.

Python resolves method and attribute lookups using the C3 linearization of the class and its parents.

class A(object):
  attr = 'A'

class B(A):
  pass
  
class C(A):
  attr = 'C'

class D(B, C):
  pass

d = D()

print(d.attr)
print(d.__class__.mro())

Output:

C
[<class '__main__.D'>, <class '__main__.B'>, <class '__main__.C'>, <class '__main__.A'>, <type 'object'>]

mro() or __mro__ is another unique function of Python which gives the method resolution order of classes in the inheritance.

What are Bound and Un-bound methods in Python?

In Python 2, when a function is defined in the body of a class, it is transformed into an unbound method.

When a function is accessed on a class instance, it is converted into a bound method, that automatically passes the instance to the method as the first self parameter.

class A(object):
  def fun(self):
    return 'Hello'
a = A()
print(A.fun)
print(a.fun)

Output:

<unbound method A.fun>
<bound method A.fun of <__main__.A object at 0x7f57e3fe84d0>>

Python 3+ has no unbound methods.