wedge

We know that objects are created in two main ways, one is through the Python/C API, the other is by calling the type object. For instance objects of built-in types, both methods are supported. For example, lists can be created through [] or list(). The former is a Python/C API and the latter is a calling type object.

But for instance objects of custom classes, we can only create them by calling type objects. If an object can be called, then the object is callable, otherwise it is not callable.

Determining whether an object is callable depends on whether a method is defined in its corresponding type object. If you look at it from a Python perspective, this method is __call__, and from an interpreter perspective, this method is tp_call.

Looking at object calls from a Python perspective

You can create an integer, string, or tuple by calling int, str, or tuple, and you can also call a custom class Create the corresponding instance object, indicating that the type object is callable, that is, callable. Then there must be a __call__ method inside the type object (type) of these type objects (int, str, tuple, class, etc.).

# int可以調(diào)用
# 那么它的類型對(duì)象、也就是元類(type), 內(nèi)部一定有__call__方法
print(hasattr(type, "__call__"))# True
# 而調(diào)用一個(gè)對(duì)象，等價(jià)于調(diào)用其類型對(duì)象的 __call__ 方法
# 所以 int(3.14)實(shí)際就等價(jià)于如下
print(type.__call__(int, 3.14))# 3

Note: The description here may be a little confusing. We say that int, str, and float are all type objects (simply speaking, classes), and 123, "Hello", and 3.14 are their corresponding Instance objects, these are fine. But is type a type object? Obviously yes, although we call it a metaclass, it is also a type object. If print(type) displays a class, it is also a type object.

So relative to type, have int, str, and float become instance objects again? Because their type is type.

So class has duality:

• If you stand from the perspective of instance objects (such as: 123, "satori", [], 3.14) , it is a type object
• If from the perspective of type, it is an instance object
  

In the same way, the type of type is also type, then type is both the type of type Object, type is also an instance object of type. Although the description here will be a bit confusing, it should not be difficult to understand, and in order to avoid ambiguity in subsequent descriptions, here we make a statement:

• Integers, floating point numbers, strings, etc. etc., we call them instance objects
  
• int, float, str, dict, and our custom classes, we call them type objects
  
• Although it is also Type object, but we call it a metaclass
  

So there is a __call__ method inside type, which means that all type objects are callable, because calling a type object is calling type_ _call__ method. Whether the instance object can be called is not necessarily determined. It depends on whether the __call__ method is defined in its type object, because calling an object essentially executes the __call__ method inside its type object.

class A:
 pass
a = A()
# 因?yàn)槲覀冏远x的類 A 里面沒(méi)有 __call__
# 所以 a 是不可以被調(diào)用的
try:
 a()
except Exception as e:
 # 告訴我們 A 的實(shí)例對(duì)象不可以被調(diào)用
 print(e)# 'A' object is not callable
# 如果我們給 A 設(shè)置了一個(gè) __call__
type.__setattr__(A, "__call__", lambda self: "這是__call__")
# 發(fā)現(xiàn)可以調(diào)用了
print(a())# 這是__call__

We see that this is the characteristic of dynamic languages. Even after the class is created, it can still be dynamically set through type, which is not supported in static languages. So type is the metaclass of all classes. It controls the generation process of our custom classes. This ancient and powerful class type can allow us to play many new tricks.

But for built-in classes, type cannot dynamically add, delete or modify attributes, because built-in classes are statically defined at the bottom level. Because we can see from the source code that these built-in classes, including metaclasses, are all PyTypeObject objects and have been declared as global variables at the bottom level, or they already exist as static classes. Therefore, although type is the metaclass of all types of objects, type only has the ability to add, delete, and modify when it comes to our custom classes.

And we have also explained that the dynamic nature of Python is dynamically assigned when the interpreter translates bytecode into C code. Therefore, dynamically setting attributes or methods for a class only applies to dynamic classes. That is, the class defined using the class keyword in the py file.

As for static classes or extension classes defined when writing extension modules (the two are equivalent), they already point to C-level data structures after compilation, and there is no need to They have been explained by the interpreter, so the interpreter naturally cannot do anything with them. After all, a strong life does not need explanation.

try:
 type.__setattr__(dict, "__call__", lambda self: "這是__call__")
except Exception as e:
 print(e)# can't set attributes of built-in/extension type 'dict'

We saw an exception being thrown, prompting us that we cannot set attributes for the built-in/extended type dict, because they bypass the interpreter interpretation and execution step, so their attributes cannot be set dynamically.

The same is true for instance objects. Instance objects of static classes cannot dynamically set attributes:

class Girl:
 pass
g = Girl()
g.name = "古明地覺(jué)"
# 實(shí)例對(duì)象我們也可以手動(dòng)設(shè)置屬性
print(g.name)# 古明地覺(jué)
lst = list()
try:
 lst.name = "古明地覺(jué)"
except Exception as e:
 # 但是內(nèi)置類型的實(shí)例對(duì)象是不可以的
 print(e)# 'list' object has no attribute 'name'

Some people may be surprised, why not lists? The answer is that the instance object of the built-in type does not have a __dict__ attribute dictionary, because the relevant attributes or methods have been defined at the bottom and cannot be added dynamically. If we set __slots__ when we customize the class, the effect is the same as the built-in class.

當(dāng)然了，我們后面會(huì)介紹如何通過(guò)動(dòng)態(tài)修改解釋器來(lái)改變這一點(diǎn)，舉個(gè)栗子，不是說(shuō)靜態(tài)類無(wú)法動(dòng)態(tài)設(shè)置屬性嗎？下面我就來(lái)打自己臉：

import gc
try:
 type.__setattr__(list, "ping", "pong")
except TypeError as e:
 print(e)# can't set attributes of built-in/extension type 'list'
# 我們看到無(wú)法設(shè)置，那么我們就來(lái)改變這一點(diǎn)
attrs = gc.get_referents(tuple.__dict__)[0]
attrs["ping"] = "pong"
print(().ping)# pong
attrs["append"] = lambda self, item: self + (item,)
print(
 ().append(1).append(2).append(3)
)# (1, 2, 3)

我臉腫了。好吧，其實(shí)這只是我們玩的一個(gè)小把戲，當(dāng)我們介紹完整個(gè) CPython 的時(shí)候，會(huì)來(lái)專門(mén)聊一聊如何動(dòng)態(tài)修改解釋器。比如：讓元組變得可修改，讓 Python 真正利用多核等等。

從解釋器的角度看對(duì)象的調(diào)用

我們以內(nèi)置類型 float 為例，我們說(shuō)創(chuàng)建一個(gè) PyFloatObject，可以通過(guò)3.14或者float(3.14)的方式。前者使用Python/C API創(chuàng)建，3.14直接被解析為 C 一級(jí)數(shù)據(jù)結(jié)構(gòu)，也就是PyFloatObject實(shí)例；后者使用類型對(duì)象創(chuàng)建，通過(guò)對(duì)float進(jìn)行一個(gè)調(diào)用、將3.14作為參數(shù)，最終也得到指向C一級(jí)數(shù)據(jù)結(jié)構(gòu)PyFloatObject實(shí)例。

Python/C API的創(chuàng)建方式我們已經(jīng)很清晰了，就是根據(jù)值來(lái)推斷在底層應(yīng)該對(duì)應(yīng)哪一種數(shù)據(jù)結(jié)構(gòu)，然后直接創(chuàng)建即可。我們重點(diǎn)看一下通過(guò)類型調(diào)用來(lái)創(chuàng)建實(shí)例對(duì)象的方式。

如果一個(gè)對(duì)象可以被調(diào)用，它的類型對(duì)象中一定要有tp_call(更準(zhǔn)確的說(shuō)成員tp_call的值是一個(gè)函數(shù)指針，不可以是0)，而PyFloat_Type是可以調(diào)用的，這就說(shuō)明PyType_Type內(nèi)部的tp_call是一個(gè)函數(shù)指針，這在Python的層面上我們已經(jīng)驗(yàn)證過(guò)了，下面我們?cè)賮?lái)通過(guò)源碼看一下。

//typeobject.c
PyTypeObject PyType_Type = {
 PyVarObject_HEAD_INIT(&PyType_Type, 0)
 "type", /* tp_name */
 sizeof(PyHeapTypeObject), /* tp_basicsize */
 sizeof(PyMemberDef),/* tp_itemsize */
 (destructor)type_dealloc, /* tp_dealloc */
 //... /* tp_hash */
 (ternaryfunc)type_call, /* tp_call */
 //...
}

我們看到在實(shí)例化PyType_Type的時(shí)候PyTypeObject內(nèi)部的成員tp_call被設(shè)置成了type_call。這是一個(gè)函數(shù)指針，當(dāng)我們調(diào)用PyFloat_Type的時(shí)候，會(huì)觸發(fā)這個(gè)type_call指向的函數(shù)。

因此 float(3.14) 在C的層面上等價(jià)于：

(&PyFloat_Type) -> ob_type -> tp_call(&PyFloat_Type, args, kwargs);
// 即：
(&PyType_Type) -> tp_call(&PyFloat_Type, args, kwargs);
// 而在創(chuàng)建 PyType_Type 的時(shí)候，給 tp_call 成員傳遞的是 type_call
// 因此最終相當(dāng)于
type_call(&PyFloat_Type, args, kwargs)

如果用 Python 來(lái)演示這一過(guò)程的話：

# float(3.14)，等價(jià)于
f1 = float.__class__.__call__(float, 3.14)
# 等價(jià)于
f2 = type.__call__(float, 3.14)
print(f1, f2)# 3.14 3.14

這就是 float(3.14) 的秘密，相信list、dict在實(shí)例化的時(shí)候是怎么做的，你已經(jīng)猜到了，做法是相同的。

# lst = list("abcd")
lst = list.__class__.__call__(list, "abcd")
print(lst)# ['a', 'b', 'c', 'd']
# dct = dict([("name", "古明地覺(jué)"), ("age", 17)])
dct = dict.__class__.__call__(dict, [("name", "古明地覺(jué)"), ("age", 17)])
print(dct)# {'name': '古明地覺(jué)', 'age': 17}

最后我們來(lái)圍觀一下 type_call 函數(shù)，我們說(shuō) type 的 __call__ 方法，在底層對(duì)應(yīng)的是 type_call 函數(shù)，它位于Object/typeobject.c中。

static PyObject *
type_call(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
 // 如果我們調(diào)用的是 float
 // 那么顯然這里的 type 就是 &PyFloat_Type

 // 這里是聲明一個(gè)PyObject *
 // 顯然它是要返回的實(shí)例對(duì)象的指針
 PyObject *obj;

 // 這里會(huì)檢測(cè) tp_new是否為空，tp_new是什么估計(jì)有人已經(jīng)猜到了
 // 我們說(shuō)__call__對(duì)應(yīng)底層的tp_call
 // 顯然__new__對(duì)應(yīng)底層的tp_new，這里是為實(shí)例對(duì)象分配空間
 if (type->tp_new == NULL) {
 // tp_new 是一個(gè)函數(shù)指針，指向具體的構(gòu)造函數(shù)
 // 如果 tp_new 為空，說(shuō)明它沒(méi)有構(gòu)造函數(shù)
 // 因此會(huì)報(bào)錯(cuò)，表示無(wú)法創(chuàng)建其實(shí)例
 PyErr_Format(PyExc_TypeError,
"cannot create '%.100s' instances",
type->tp_name);
 return NULL;
 }

 //通過(guò)tp_new分配空間
 //此時(shí)實(shí)例對(duì)象就已經(jīng)創(chuàng)建完畢了，這里會(huì)返回其指針
 obj = type->tp_new(type, args, kwds);
 //類型檢測(cè)，暫時(shí)不用管
 obj = _Py_CheckFunctionResult((PyObject*)type, obj, NULL);
 if (obj == NULL)
 return NULL;
 //我們說(shuō)這里的參數(shù)type是類型對(duì)象，但也可以是元類
 //元類也是由PyTypeObject結(jié)構(gòu)體實(shí)例化得到的
 //元類在調(diào)用的時(shí)候執(zhí)行的依舊是type_call
 //所以這里是檢測(cè)type指向的是不是PyType_Type
 //如果是的話，那么實(shí)例化得到的obj就不是實(shí)例對(duì)象了，而是類型對(duì)象
 //要單獨(dú)檢測(cè)一下
 if (type == &PyType_Type &&
 PyTuple_Check(args) && PyTuple_GET_SIZE(args) == 1 &&
 (kwds == NULL ||
(PyDict_Check(kwds) && PyDict_GET_SIZE(kwds) == 0)))
 return obj;
 //tp_new應(yīng)該返回相應(yīng)類型對(duì)象的實(shí)例對(duì)象(的指針)
 //但如果不是，就直接將這里的obj返回
 //此處這么做可能有點(diǎn)難理解，我們一會(huì)細(xì)說(shuō)
 if (!PyType_IsSubtype(Py_TYPE(obj), type))
 return obj;

 //拿到obj的類型
 type = Py_TYPE(obj);
 //執(zhí)行 tp_init
 //顯然這個(gè)tp_init就是__init__函數(shù)
 //這與Python中類的實(shí)例化過(guò)程是一致的。
 if (type->tp_init != NULL) {
 //將tp_new返回的對(duì)象作為self，執(zhí)行 tp_init
 int res = type->tp_init(obj, args, kwds);
 if (res < 0) {
 //執(zhí)行失敗，將引入計(jì)數(shù)減1，然后將obj設(shè)置為NULL
 assert(PyErr_Occurred());
 Py_DECREF(obj);
 obj = NULL;
 }
 else {
 assert(!PyErr_Occurred());
 }
 }
 //返回obj
 return obj;
}

因此從上面我們可以看到關(guān)鍵的部分有兩個(gè)：

• 調(diào)用類型對(duì)象的 tp_new 指向的函數(shù)為實(shí)例對(duì)象申請(qǐng)內(nèi)存
  
• 調(diào)用 tp_init 指向的函數(shù)為實(shí)例對(duì)象進(jìn)行初始化，也就是設(shè)置屬性
  

所以這對(duì)應(yīng)Python中的__new__和__init__，我們說(shuō)__new__是為實(shí)例對(duì)象開(kāi)辟一份內(nèi)存，然后返回指向這片內(nèi)存(對(duì)象)的指針，并且該指針會(huì)自動(dòng)傳遞給__init__中的self。

class Girl:
 def __new__(cls, name, age):
 print("__new__方法執(zhí)行啦")
 # 寫(xiě)法非常固定
 # 調(diào)用object.__new__(cls)就會(huì)創(chuàng)建Girl的實(shí)例對(duì)象
 # 因此這里的cls指的就是這里的Girl，注意：一定要返回
 # 因?yàn)開(kāi)_new__會(huì)將自己的返回值交給__init__中的self
 return object.__new__(cls)
 def __init__(self, name, age):
 print("__init__方法執(zhí)行啦")
 self.name = name
 self.age = age
g = Girl("古明地覺(jué)", 16)
print(g.name, g.age)
"""
__new__方法執(zhí)行啦
__init__方法執(zhí)行啦
古明地覺(jué) 16
"""

__new__里面的參數(shù)要和__init__里面的參數(shù)保持一致，因?yàn)槲覀儠?huì)先執(zhí)行__new__，然后解釋器會(huì)將__new__的返回值和我們傳遞的參數(shù)組合起來(lái)一起傳遞給__init__。因此__new__里面的參數(shù)除了cls之外，一般都會(huì)寫(xiě)*args和**kwargs。

然后再回過(guò)頭來(lái)看一下type_call中的這幾行代碼：

static PyObject *
type_call(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
 //......
 //......
 if (!PyType_IsSubtype(Py_TYPE(obj), type))
 return obj;

 //......
 //......
}

我們說(shuō)tp_new應(yīng)該返回該類型對(duì)象的實(shí)例對(duì)象，而且一般情況下我們是不寫(xiě)__new__的，會(huì)默認(rèn)執(zhí)行。但是我們一旦重寫(xiě)了，那么必須要手動(dòng)返回object.__new__(cls)。可如果我們不返回，或者返回其它的話，會(huì)怎么樣呢？

class Girl:
 def __new__(cls, *args, **kwargs):
 print("__new__方法執(zhí)行啦")
 instance = object.__new__(cls)
 # 打印看看instance到底是個(gè)什么東東
 print("instance:", instance)
 print("type(instance):", type(instance))

 # 正確做法是將instance返回
 # 但是我們不返回, 而是返回個(gè) 123
 return 123
 def __init__(self, name, age):
 print("__init__方法執(zhí)行啦")
g = Girl()
"""
__new__方法執(zhí)行啦
instance: <__main__.Girl object at 0x000002C0F16FA1F0>
type(instance): <class '__main__.Girl'>
"""

這里面有很多可以說(shuō)的點(diǎn)，首先就是 __init__ 里面需要兩個(gè)參數(shù)，但是我們沒(méi)有傳，卻還不報(bào)錯(cuò)。原因就在于這個(gè) __init__ 壓根就沒(méi)有執(zhí)行，因?yàn)?__new__ 返回的不是 Girl 的實(shí)例對(duì)象。

通過(guò)打印 instance，我們知道了object.__new__(cls) 返回的就是 cls 的實(shí)例對(duì)象，而這里的cls就是Girl這個(gè)類本身。我們必須要返回instance，才會(huì)執(zhí)行對(duì)應(yīng)的__init__，否則__new__直接就返回了。我們?cè)谕獠縼?lái)打印一下創(chuàng)建的實(shí)例對(duì)象吧，看看結(jié)果：

class Girl:
 def __new__(cls, *args, **kwargs):
 return 123
 def __init__(self, name, age):
 print("__init__方法執(zhí)行啦")
g = Girl()
print(g, type(g))# 123 <class 'int'>

我們看到打印的是123，所以再次總結(jié)一些tp_new和tp_init之間的區(qū)別，當(dāng)然也對(duì)應(yīng)__new__和__init__的區(qū)別：

• tp_new：為該類型對(duì)象的實(shí)例對(duì)象申請(qǐng)內(nèi)存，在Python的__new__方法中通過(guò)object.__new__(cls)的方式申請(qǐng)，然后將其返回
• tp_init：tp_new的返回值會(huì)自動(dòng)傳遞給self，然后為self綁定相應(yīng)的屬性，也就是進(jìn)行實(shí)例對(duì)象的初始化
  

但如果tp_new返回的不是對(duì)應(yīng)類型的實(shí)例對(duì)象的指針，比如type_call中第一個(gè)參數(shù)接收的&PyFloat_Type，但是tp_new中返回的卻是PyLongObject *，所以此時(shí)就不會(huì)執(zhí)行tp_init。

以上面的代碼為例，我們Girl中的__new__應(yīng)該返回Girl的實(shí)例對(duì)象才對(duì)，但實(shí)際上返回了整型，因此類型不一致，所以不會(huì)執(zhí)行__init__。

下面我們可以做總結(jié)了，通過(guò)類型對(duì)象去創(chuàng)建實(shí)例對(duì)象的整體流程如下：

• 第一步：獲取類型對(duì)象的類型對(duì)象，說(shuō)白了就是元類，執(zhí)行元類的 tp_call 指向的函數(shù)，即 type_call
  
• 第二步：type_call 會(huì)調(diào)用該類型對(duì)象的 tp_new 指向的函數(shù)，如果 tp_new 為 NULL，那么會(huì)到 tp_base 指定的父類里面去尋找 tp_new。在新式類當(dāng)中，所有的類都繼承自 object，因此最終會(huì)執(zhí)行 object 的 __new__。然后通過(guò)訪問(wèn)對(duì)應(yīng)類型對(duì)象中的 tp_basicsize 信息，這個(gè)信息記錄著該對(duì)象的實(shí)例對(duì)象需要占用多大的內(nèi)存，繼而完成申請(qǐng)內(nèi)存的操作
  
• 調(diào)用type_new 創(chuàng)建完對(duì)象之后，就會(huì)進(jìn)行實(shí)例對(duì)象的初始化，會(huì)將指向這片空間的指針交給 tp_init，但前提是 tp_new 返回的實(shí)例對(duì)象的類型要一致。
  

所以都說(shuō) Python 在實(shí)例化的時(shí)候會(huì)先調(diào)用 __new__ 方法，再調(diào)用 __init__ 方法，相信你應(yīng)該知道原因了，因?yàn)樵谠创a中先調(diào)用 tp_new、再調(diào)用的 tp_init。

static PyObject *
type_call(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
 //調(diào)用__new__方法， 拿到其返回值
 obj = type->tp_new(type, args, kwds);
 if (type->tp_init != NULL) {
 //將__new__返回的實(shí)例obj，和args、kwds組合起來(lái)
 //一起傳給 __init__
 //其中 obj 會(huì)傳給 self，
 int res = type->tp_init(obj, args, kwds);
 //......
 return obj;
}

所以源碼層面表現(xiàn)出來(lái)的，和我們?cè)?Python 層面看到的是一樣的。

小結(jié)

到此，我們就從 Python 和解釋器兩個(gè)層面了解了對(duì)象是如何調(diào)用的，更準(zhǔn)確的說(shuō)我們是從解釋器的角度對(duì) Python 層面的知識(shí)進(jìn)行了驗(yàn)證，通過(guò) tp_new 和 tp_init 的關(guān)系，來(lái)了解 __new__ 和 __init__ 的關(guān)系。

另外，對(duì)象調(diào)用遠(yuǎn)不止我們目前說(shuō)的這么簡(jiǎn)單，更多的細(xì)節(jié)隱藏在了幕后，只不過(guò)現(xiàn)在沒(méi)辦法將其一次性全部挖掘出來(lái)。

The above is the detailed content of Source code exploration: How are objects called in Python?. For more information, please follow other related articles on the PHP Chinese website!