In a recent phone screen, I decided to use a class attribute in my implementation of a certain Python API. My interviewer challenged me, questioning whether my code was syntactically valid, when it was executed, etc. In fact, I wasn’t sure of the answers myself. So I did some digging. Python class attributes: when (or how) to use them. In this guide, I walk you through common pitfalls and conclude with a list of valid use cases that could save you time, energy, and lines of code. Show
Share ShareIn a recent phone screen, I decided to use a class attribute in my implementation of a certain Python API. My interviewer challenged me, questioning whether my code was syntactically valid, when it was executed, etc. In fact, I wasn’t sure of the answers myself. So I did some digging. Python class attributes: when (or how) to use them. In this guide, I walk you through common pitfalls and conclude with a list of valid use cases that could save you time, energy, and lines of code. By Charles MarshCharles has been an engineering lead at Khan Academy, Cedar, and Spring Discovery. Editor’s note: This article was updated on 11/29/22 by our editorial team. It has been modified to include recent sources and to align with our current editorial standards. I recently had a programming interview phone screen in which we used a collaborative text editor and it got me thinking about Python class attributes. I was asked to implement a certain API and chose to do so in Python. Abstracting away the problem statement, let’s say I needed a class whose instances stored some 3 and some 4.I took a deep breath and started typing. After a few lines, I had something like this:
My interviewer stopped me:
For reference, and to give you an idea of what I was going for, here’s how I amended the code:
As it turns out, we were both wrong. The real answer lay in understanding the distinction between Python class attributes and Python instance attributes. Note: If you have an expert handle on Python class attributes, you can skip ahead to use cases. Python Class AttributesMy interviewer was wrong in that the above code is syntactically valid. I was wrong in that the code isn’t setting a “default value” for the instance attribute. Instead, it’s defining 3 as a class attribute with value 7.In my experience, Python class attributes are a topic that many people know something about, but few understand completely. Python Class Variables vs. Instance Variables: What’s the Difference?A Python class attribute is an attribute of the class (circular, I know), rather than an attribute of an instance of a class. Let’s use a Python class example to illustrate the difference. Here, 8 is a class attribute, and 9 is an instance attribute:
Note that all instances of the class have access to 8, and that it can also be accessed as a property of the class itself:
For Java or C++ programmers, the class attribute is similar—but not identical—to the static member. We’ll see how they differ later. Python Class Properties vs. Instance NamespacesTo understand what’s happening here, let’s talk briefly about Python namespaces. A namespace is a mapping from names to objects, with the property that there is zero relation between names in different namespaces. They’re usually implemented as Python dictionaries, although this is abstracted away. Depending on the context, you may need to access a namespace using dot syntax (e.g., 1) or as a local variable (e.g., 2). As a concrete example:
Python classes and instances of classes each have their own distinct namespaces represented by the 3 and 4, respectively.When you try to access Python attributes from an instance of a class, it first looks at its instance namespace. If it finds the attribute, it returns the associated value. If not, it then looks in the class namespace and returns the attribute (if it’s present, otherwise throwing an error). For example:
The instance namespace takes precedence over the class namespace: If there is a Python attribute with the same name in both, the instance namespace will be checked first and its value returned. Here’s a for attribute lookup:
And, in visual form: How Python Class Attributes Handle AssignmentWith this in mind, we can make sense of how class attributes handle assignment:
At the namespace level we’re setting 5. (Note: This isn’t the exact code which would be 6 as 7 returns a , an immutable wrapper that prevents direct assignment, but it helps for demonstration’s sake). Then, when we access 8, 8 has a new value in the class namespace and thus 2 is returned.
At the namespace level we’re adding the 8 attribute to 1, so when we look up 8, we return 2. Meanwhile, other instances of 3 will not have 8 in their instance namespaces, so they continue to find 8 in 3 and thus return 1.MutabilityWhat if your class attribute has a mutable type? You can manipulate the class attribute by accessing it through a particular instance and, in turn, end up manipulating the referenced object that all instances are accessing (as pointed out by Timothy Wiseman). Let’s go back to the 7 I defined earlier and see how my use of a class variable could have led to problems down the road:
My goal was to have the empty list ( 7) as the default value for 3, and for each instance of 7 to have its own data that would be altered over time on an instance-by-instance basis. But in this case, we got the following behavior (note that 4 is arbitrary in this example): 0This is no good—altering our Python class variable via one instance alters it for all the others! At the namespace level all instances of 7 are accessing and modifying the same list in 3 without making their own 3 attributes in their instance namespaces.We could get around this using assignment; that is, instead of exploiting the list’s mutability, we could assign our 7 objects to have their own lists, as follows: 1In this case, we’re adding 6, so the original 7 remains unchanged.Unfortunately, this requires that 7 users have intimate knowledge of its variables, and this workaround is certainly prone to mistakes. In a sense, we’d be addressing the symptoms rather than the cause. It’s preferable to have something that is correct by construction.My personal solution: If you’re just using a class variable to assign a default value to a would-be Python instance variable, don’t use mutable values. In this case, every instance of 7 was going to override 0 with its own instance attribute eventually, so using an empty list as the default led to a tiny bug that was easily overlooked. Instead of the above, we could have either:
For example: 2Of course, we’d have to handle the 1 case appropriately, but that’s a small price to pay.When Should You Use Python Class Attributes?Class attributes are tricky, but let’s look at a few cases where they would come in handy:
Related: Python Best Practices and Tips by Toptal Developers Under the HoodNote: If you’re worrying about performance at this level, you might not want to be use Python in the first place, as the differences will be on the order of tenths of a millisecond—but it’s still fun to poke around a bit and it helps for illustration’s sake. Recall that a class’s namespace is created and filled in at the time of the class’s definition. That means that we do just one assignment for a given class variable, while instance variables must be assigned every time a new instance is created. Let’s take an example: 8We assign to 8 just once, but 9 on every call to 0.As further evidence, let’s use the Python disassembler: 9When we look at the byte code, it’s again obvious that 1 has to do two assignments, while 2 does just one.In practice, what does this gain really look like? I’ll be the first to admit that timing tests are highly dependent on often uncontrollable factors and the differences between them are often hard to explain accurately. However, I think these small snippets (run with the Python timeit module) help to illustrate the differences between class and instance variables, so I’ve included them anyway. Note: I’m on a MacBook Pro with OS X 10.8.5 and Python 2.7.2. Initialization 0The initializations of 3 are faster by over a second, so the difference here does appear to be statistically significant.So why is this the case? One speculative explanation: We do two assignments in 1, but just one in 2.Assignment 1Note: There’s no way to re-run your setup code on each trial with timeit, so we have to reinitialize our variable on our trial. The second line of times represents the above times with the previously calculated initialization times deducted. From the above, it looks like 6 only takes about 60% as long as 3 to handle assignments.Why is this the case? One speculative explanation: When we assign to 8, we first look in the instance namespace ( 9), fail to find 0, and then look in the class namespace ( 1), then make the proper assignment. When we assign to 2, we do half as many lookups, as we immediately assign to the instance namespace ( 3).In summary, though these performance gains won’t matter in reality, these tests are interesting at the conceptual level. If anything, I hope these differences help illustrate the mechanical distinctions between class and instance variables. In ConclusionClass attributes seem to be underused in Python; a lot of programmers have different impressions of how they work and why they might be helpful. My take: Python class variables have their place within the school of good code. When used with care, they can simplify things and improve readability. But when carelessly thrown into a given class, they’re sure to trip you up. Appendix: Private Instance VariablesOne additional variable to mention: private instance variables. Python doesn’t have private variables so to speak, but another interesting relationship between class and instance naming comes with name mangling. In the Python style guide, it’s said that pseudo-private variables should be prefixed with a double underscore: ‘__’. This is not only a sign to others that your variable is meant to be treated privately, but also a way to prevent access to it, of sorts. Here’s what I mean: 2Look at that: The instance attribute 4 is automatically prefixed with the class name to yield 5.While still settable and gettable using 6, this name mangling is a means of creating a “private” variable as it prevents you and others from accessing it by accident or through ignorance.Edit: As Pedro Werneck kindly pointed out, this behavior is largely intended to help out with subclassing. In the , they see it as serving two purposes: (1) preventing subclasses from accessing certain attributes, and (2) preventing namespace clashes in these subclasses. While useful, variable mangling shouldn’t be seen as an invitation to write code with an assumed public-private distinction, such as is present in Java. Further Reading on the Toptal Engineering Blog:
A Python namespace is a mapping from names to objects, with the property that there is zero relation between names in different namespaces. Namespaces are usually implemented as Python dictionaries, although this is abstracted away. In Python, a class method is a method that is invoked with the class as the context. This is often called a static method in other programming languages. An instance method, on the other hand, is invoked with an instance as the context. In that case, the instance namespace takes precedence over the class namespace. If there is an attribute with the same name in both, the instance namespace will be checked first and its value returned. |