wip

README.md

@@ -1,12 +1,14 @@
 <img src="http://mypy-lang.org/static/mypy_light.svg" alt="mypy logo" width="300px"/>
 
-# Typesafe Django Framework
+# pep484 stubs for Django framework
 
 [![Build Status](https://travis-ci.com/typeddjango/django-stubs.svg?branch=master)](https://travis-ci.com/typeddjango/django-stubs)
 [![Checked with mypy](http://www.mypy-lang.org/static/mypy_badge.svg)](http://mypy-lang.org/)
 [![Gitter](https://badges.gitter.im/mypy-django/Lobby.svg)](https://gitter.im/mypy-django/Lobby)
 
-This package contains [type stubs](https://www.python.org/dev/peps/pep-0561/) and a custom mypy plugin to provide more precise static types and type inference for Django framework. Django uses some Python "magic" that makes having precise types for some code patterns problematic. This is why we need this project. The final goal is to be able to get precise types for most common patterns.
+This package contains type stubs and mypy plugin to provide more precise static types and type inference for Django framework. Django uses some Python "magic" that makes having precise types for some code patterns problematic. This is why we need to accompany the stubs with mypy plugins. The final goal is to be able to get precise types for most common patterns.
+
+Could be run on earlier versions of Django, but expect some missing imports warnings.
 
 
 ## Installation
@@ -15,35 +17,8 @@ This package contains [type stubs](https://www.python.org/dev/peps/pep-0561/) an
 pip install django-stubs
 ```
 
-See [Configutation](#configuration) section to get started.
 
-
+## Mypy compatibility
-## Configuration
-
-To make `mypy` happy, you will need to add:
-
-```ini
-[mypy]
-plugins =
-    mypy_django_plugin.main
-    
-[mypy.plugins.django-stubs]
-django_settings_module = "myproject.settings"
-```
-
-in your `mypy.ini` or `setup.cfg` [file](https://mypy.readthedocs.io/en/latest/config_file.html).
-
-Two things happeining here:
-
-1. We need to explicitly list our plugin to be loaded by `mypy`
-2. Our plugin also requires `django` settings module (what you put into `DJANGO_SETTINGS_MODULE` variable) to be specified
-
-This fully working [typed boilerplate](https://github.com/wemake-services/wemake-django-template) can serve you as an example.
-
-
-## Version compatibility
-
-We rely on different `django` and `mypy` versions:
 
 | django-stubs | mypy version | django version | python version
 | ------------ | ---- | ---- | ---- |
@@ -53,66 +28,58 @@ We rely on different `django` and `mypy` versions:
 | 0.12.x | old semantic analyzer (<0.711), dmypy support | 2.1.x | ^3.6
 
 
-## FAQ
+## Configuration
 
-> Is this an official Django project?
+To make mypy aware of the plugin, you need to add
 
-No, it is not. We are indendepent from Django at the moment.
+```ini
-There's a [proposal](https://github.com/django/deps/pull/65) to merge our project into the Django itself.
+[mypy]
-You show your support by linking the PR.
+plugins =
+    mypy_django_plugin.main
+```
 
-> Is it safe to use this in production?
+in your `mypy.ini` or `setup.cfg` [file](https://mypy.readthedocs.io/en/latest/config_file.html).
 
-Yes, it is! This project does not affect your runtime at all.
+Plugin also requires Django settings module (what you put into `DJANGO_SETTINGS_MODULE` variable) to be specified.
-It only affects `mypy` type checking process.
 
-But, it does not make sense to use this project without `mypy`.
+```ini
+[mypy]
+strict_optional = True
 
-> mypy crashes when I run it with this plugin installed
+# This one is new:
+[mypy.plugins.django-stubs]
+django_settings_module = mysettings
+```
+
+Where `mysettings` is a value of `DJANGO_SETTINGS_MODULE` (with or without quotes)
 
 Current implementation uses Django runtime to extract models information, so it will crash, if your installed apps `models.py` is not correct. For this same reason, you cannot use `reveal_type` inside global scope of any Python file that will be executed for `django.setup()`. 
 
 In other words, if your `manage.py runserver` crashes, mypy will crash too. 
-You can also run `mypy` with [`--tb`](https://mypy.readthedocs.io/en/stable/command_line.html#cmdoption-mypy-show-traceback)
-option to get extra information about the error.
 
-> I cannot use QuerySet or Manager with type annotations
+This fully working [typed boilerplate](https://github.com/wemake-services/wemake-django-template) can serve you as an example.
 
-You can get a `TypeError: 'type' object is not subscriptable` 
-when you will try to use `QuerySet[MyModel]` or `Manager[MyModel]`.
 
-This happens because Django classes do not support [`__class_getitem__`](https://www.python.org/dev/peps/pep-0560/#class-getitem) magic method.
+## Notes
 
-There are several things you can use strings instead: `'QuerySet[MyModel]'` and `'Manager[MyModel'`, this way it will work as a type for `mypy` and as a regular `str` in runtime.
+Type implementation monkey-patches Django to add `__class_getitem__` to the `Manager` class. 
-
+If you would use Python3.7 and do that too in your code, you can make things like
-Currently we [are working](https://github.com/django/django/pull/12405) on providing `__class_getitem__` to the classes where we need them.
-
-> How can I use HttpRequest with custom user model?
-
-You can subclass standard request like so:
 
 ```python
-from django.http import HttpRequest
+class MyUserManager(models.Manager['MyUser']):
-from my_user_app.models import MyUser
+    pass
 
-class MyRequest(HttpRequest):
+class MyUser(models.Model):
-    user: MyUser
+    objects = MyUserManager()
 ```
 
-And then use `MyRequest` instead of standard `HttpRequest` inside your project.
+work, which should make a error messages a bit better. 
-
-
-## Related projects
-
-- [`awesome-python-typing`](https://github.com/typeddjango/awesome-python-typing) - Awesome list of all typing-related things in Python.
-- [`djangorestframework-stubs`](https://github.com/typeddjango/djangorestframework-stubs) - Stubs for Django REST Framework.
-- [`pytest-mypy-plugins`](https://github.com/typeddjango/pytest-mypy-plugins) - `pytest` plugin that we use for testing `mypy` stubs and plugins.
-- [`wemake-django-template`](https://github.com/wemake-services/wemake-django-template) - Create new typed Django projects in seconds.
 
+Otherwise, custom type will be created in mypy, named `MyUser__MyUserManager`, which will rewrite base manager as `models.Manager[User]` to make methods like `get_queryset()` and others return properly typed `QuerySet`. 
 
 
 ## To get help
 
 We have Gitter here: <https://gitter.im/mypy-django/Lobby>
 
-If you think you have more generic typing issue, please refer to <https://github.com/python/mypy> and their Gitter.
+If you think you have more generic typing issue, please refer to https://github.com/python/mypy and their Gitter.

django-stubs/conf/urls/__init__.pyi

@@ -5,10 +5,10 @@ from django.http.response import HttpResponse, HttpResponseBase
 
 from django.urls import URLResolver, URLPattern
 
-handler400: Union[str, Callable[..., HttpResponse]] = ...
+handler400: Callable[..., HttpResponse] = ...
-handler403: Union[str, Callable[..., HttpResponse]] = ...
+handler403: Callable[..., HttpResponse] = ...
-handler404: Union[str, Callable[..., HttpResponse]] = ...
+handler404: Callable[..., HttpResponse] = ...
-handler500: Union[str, Callable[..., HttpResponse]] = ...
+handler500: Callable[..., HttpResponse] = ...
 
 IncludedURLConf = Tuple[List[URLResolver], Optional[str], Optional[str]]
 

django-stubs/core/checks/__init__.pyi

@@ -1,17 +1,5 @@
-from .messages import (
+from .messages import Warning as Warning, Info as Info, Debug as Debug, Error as Error, Critical as Critical
-    CheckMessage as CheckMessage,
-    Debug as Debug,
-    Info as Info,
-    Warning as Warning,
-    Error as Error,
-    Critical as Critical,
-    DEBUG as DEBUG,
-    INFO as INFO,
-    WARNING as WARNING,
-    ERROR as ERROR,
-    CRITICAL as CRITICAL,
-)
 
-from .registry import register as register, run_checks as run_checks, tag_exists as tag_exists, Tags as Tags
+from .registry import run_checks as run_checks, Tags as Tags, register as register
 
 from . import model_checks as model_checks

django-stubs/core/checks/messages.pyi

@@ -7,11 +7,11 @@ ERROR: int
 CRITICAL: int
 
 class CheckMessage:
-    level: int = ...
+    level: Any = ...
-    msg: str = ...
+    msg: Any = ...
-    hint: Optional[str] = ...
+    hint: Any = ...
     obj: Any = ...
-    id: Optional[str] = ...
+    id: Any = ...
     def __init__(
         self, level: int, msg: str, hint: Optional[str] = ..., obj: Any = ..., id: Optional[str] = ...
     ) -> None: ...
@@ -25,9 +25,19 @@ class Info(CheckMessage):
     def __init__(self, *args: Any, **kwargs: Any) -> None: ...
 
 class Warning(CheckMessage):
+    hint: str
+    id: str
+    level: int
+    msg: str
+    obj: Any
     def __init__(self, *args: Any, **kwargs: Any) -> None: ...
 
 class Error(CheckMessage):
+    hint: None
+    id: str
+    level: int
+    msg: str
+    obj: Any
     def __init__(self, *args: Any, **kwargs: Any) -> None: ...
 
 class Critical(CheckMessage):

django-stubs/core/files/base.pyi

@@ -8,7 +8,7 @@ _T = TypeVar("_T", bound="File")
 
 class File(FileProxyMixin, IO[Any]):
     DEFAULT_CHUNK_SIZE: Any = ...
-    file: IO[Any] = ...
+    file: StringIO = ...
     name: str = ...
     mode: str = ...
     def __init__(self, file: Any, name: Optional[str] = ...) -> None: ...

django-stubs/core/management/base.pyi

@@ -23,16 +23,13 @@ class OutputWrapper(TextIOBase):
     @property
     def style_func(self): ...
     @style_func.setter
-    def style_func(self, style_func: Callable[[str], str]): ...
+    def style_func(self, style_func: Any): ...
     ending: str = ...
     def __init__(
-        self, out: Union[StringIO, TextIOWrapper], style_func: Optional[Callable[[str], str]] = ..., ending: str = ...
+        self, out: Union[StringIO, TextIOWrapper], style_func: Optional[Callable] = ..., ending: str = ...
     ) -> None: ...
     def __getattr__(self, name: str) -> Callable: ...
     def isatty(self) -> bool: ...
-    def write(  # type: ignore[override]
-        self, msg: str, style_func: Optional[Callable[[str], str]] = ..., ending: Optional[str] = ...
-    ) -> None: ...
 
 class BaseCommand:
     help: str = ...

django-stubs/db/models/fields/__init__.pyi

@@ -42,6 +42,9 @@ _ST = TypeVar("_ST")
 # __get__ return type
 _GT = TypeVar("_GT")
 
+class CharField(Field[str, str]):
+
+
 class Field(RegisterLookupMixin, Generic[_ST, _GT]):
     _pyi_private_set_type: Any
     _pyi_private_get_type: Any

django-stubs/db/models/manager.pyi

@@ -13,7 +13,6 @@ class BaseManager(QuerySet[_T]):
     name: str = ...
     model: Type[_T] = ...
     db: str
-    _db: Optional[str]
     def __init__(self) -> None: ...
     def deconstruct(self) -> Tuple[bool, str, None, Tuple, Dict[str, int]]: ...
     def check(self, **kwargs: Any) -> List[Any]: ...

django-stubs/utils/cache.pyi

@@ -1,6 +1,7 @@
 from typing import Any, Optional, Tuple
 
 from django.core.cache.backends.base import BaseCache
+from django.core.cache.backends.locmem import LocMemCache
 from django.core.handlers.wsgi import WSGIRequest
 from django.http.response import HttpResponse, HttpResponseBase
 
@@ -27,5 +28,5 @@ def learn_cache_key(
     response: HttpResponse,
     cache_timeout: Optional[float] = ...,
     key_prefix: Optional[str] = ...,
-    cache: Optional[BaseCache] = ...,
+    cache: Optional[LocMemCache] = ...,
 ) -> str: ...

django-stubs/utils/functional.pyi

@@ -1,21 +1,16 @@
-from typing import Any, Callable, Dict, List, Optional, Tuple, Type, Union, TypeVar, Generic, overload
+from typing import Any, Callable, Dict, List, Optional, Tuple, Type, Union
 from functools import wraps as wraps  # noqa: F401
 
 from django.db.models.base import Model
 
 def curry(_curried_func: Any, *args: Any, **kwargs: Any): ...
 
-_T = TypeVar("_T")
+class cached_property:
-
+    func: Callable = ...
-class cached_property(Generic[_T]):
-    func: Callable[..., _T] = ...
     __doc__: Any = ...
     name: str = ...
-    def __init__(self, func: Callable[..., _T], name: Optional[str] = ...): ...
+    def __init__(self, func: Callable, name: Optional[str] = ...) -> None: ...
-    @overload
+    def __get__(self, instance: Any, cls: Type[Any] = ...) -> Any: ...
-    def __get__(self, instance: None, cls: Type[Any] = ...) -> "cached_property[_T]": ...
-    @overload
-    def __get__(self, instance: object, cls: Type[Any] = ...) -> _T: ...
 
 class Promise: ...
 

mfile.py

@@ -0,0 +1,113 @@
+from graphviz import Digraph
+from mypy.options import Options
+
+source = """
+from root.package import MyQuerySet 
+
+MyQuerySet().mymethod()
+"""
+
+from mypy import parse
+
+parsed = parse.parse(source, 'myfile.py', None, None, Options())
+print(parsed)
+
+graphattrs = {
+    "labelloc": "t",
+    "fontcolor": "blue",
+    # "bgcolor": "#333333",
+    "margin": "0",
+}
+
+nodeattrs = {
+    # "color": "white",
+    "fontcolor": "#00008b",
+    # "style": "filled",
+    # "fillcolor": "#ffffff",
+    # "fillcolor": "#006699",
+}
+
+edgeattrs = {
+    # "color": "white",
+    # "fontcolor": "white",
+}
+
+graph = Digraph('mfile.py', graph_attr=graphattrs, node_attr=nodeattrs, edge_attr=edgeattrs)
+graph.node('__builtins__')
+
+graph.node('django.db.models')
+graph.node('django.db.models.fields')
+
+graph.edge('django.db.models', 'django.db.models.fields')
+graph.edge('django.db.models', '__builtins__')
+graph.edge('django.db.models.fields', '__builtins__')
+
+graph.node('mymodule')
+graph.edge('mymodule', 'django.db.models')
+graph.edge('mymodule', '__builtins__')
+#
+# graph.node('ImportFrom', label='ImportFrom(val=root.package, [MyQuerySet])')
+# graph.edge('MypyFile', 'ImportFrom')
+
+
+
+# graph.node('ClassDef_MyQuerySet', label='ClassDef(name=MyQuerySet)')
+# graph.edge('MypyFile', 'ClassDef_MyQuerySet')
+#
+# graph.node('FuncDef_mymethod', label='FuncDef(name=mymethod)')
+# graph.edge('ClassDef_MyQuerySet', 'FuncDef_mymethod')
+#
+# graph.node('Args', label='Args')
+# graph.edge('FuncDef_mymethod', 'Args')
+#
+# graph.node('Var_self', label='Var(name=self)')
+# graph.edge('Args', 'Var_self')
+#
+# graph.node('Block', label='Block')
+# graph.edge('FuncDef_mymethod', 'Block')
+#
+# graph.node('PassStmt')
+# graph.edge('Block', 'PassStmt')
+
+# graph.node('ExpressionStmt')
+# graph.edge('MypyFile', 'ExpressionStmt')
+#
+# graph.node('CallExpr', label='CallExpr(val="MyQuerySet()")')
+# graph.edge('ExpressionStmt', 'CallExpr')
+#
+# graph.node('MemberExpr', label='MemberExpr(val=".mymethod()")')
+# graph.edge('CallExpr', 'MemberExpr')
+#
+# graph.node('CallExpr_outer_Args', label='Args()')
+# graph.edge('CallExpr', 'CallExpr_outer_Args')
+#
+# graph.node('CallExpr_inner', label='CallExpr(val="mymethod()")')
+# graph.edge('MemberExpr', 'CallExpr_inner')
+#
+# graph.node('NameExpr', label='NameExpr(val="mymethod")')
+# graph.edge('CallExpr_inner', 'NameExpr')
+#
+# graph.node('Expression_Args', label='Args()')
+# graph.edge('CallExpr_inner', 'Expression_Args')
+
+graph.render(view=True, format='png')
+
+
+# MypyFile(
+#   ClassDef(
+#     name=MyQuerySet,
+#     FuncDef(
+#       name=mymethod,
+#       Args(
+#         Var(self))
+#       Block(PassStmt())
+#     )
+# )
+# ExpressionStmt:6(
+#   CallExpr:6(
+#     MemberExpr:6(
+#       CallExpr:6(
+#         NameExpr(MyQuerySet)
+#         Args())
+#       mymethod)
+#     Args())))

mfile.py.gv

@@ -0,0 +1,13 @@
+digraph "mfile.py" {
+	graph [fontcolor=blue labelloc=t margin=0]
+	node [fontcolor="#00008b"]
+	__builtins__
+	"django.db.models"
+	"django.db.models.fields"
+	"django.db.models" -> "django.db.models.fields"
+	"django.db.models" -> __builtins__
+	"django.db.models.fields" -> __builtins__
+	mymodule
+	mymodule -> "django.db.models"
+	mymodule -> __builtins__
+}

my.gv

@@ -0,0 +1,9 @@
+digraph AST {
+	File
+	ClassDef
+	ClassDef -> File
+	FuncDef
+	FuncDef -> ClassDef
+	ExpressionStmt
+	ExpressionStmt -> File
+}

mypy_django_plugin/django/context.py

@@ -11,7 +11,7 @@ from django.db import models
 from django.db.models.base import Model
 from django.db.models.fields import AutoField, CharField, Field
 from django.db.models.fields.related import ForeignKey, RelatedField
-from django.db.models.fields.reverse_related import ForeignObjectRel
+from django.db.models.fields.reverse_related import ForeignObjectRel, ManyToOneRel, ManyToManyRel, OneToOneRel
 from django.db.models.lookups import Exact
 from django.db.models.sql.query import Query
 from django.utils.functional import cached_property
@@ -21,7 +21,7 @@ from mypy.types import AnyType, Instance
 from mypy.types import Type as MypyType
 from mypy.types import TypeOfAny, UnionType
 
-from mypy_django_plugin.lib import fullnames, helpers
+from mypy_django_plugin.lib import chk_helpers, fullnames, helpers
 
 try:
     from django.contrib.postgres.fields import ArrayField
@@ -55,10 +55,10 @@ def initialize_django(settings_module: str) -> Tuple['Apps', 'LazySettings']:
         def noop_class_getitem(cls, key):
             return cls
 
-        from django.db import models
+        # from django.db import models
 
-        models.QuerySet.__class_getitem__ = classmethod(noop_class_getitem)  # type: ignore
+        # models.QuerySet.__class_getitem__ = classmethod(noop_class_getitem)  # type: ignore
-        models.Manager.__class_getitem__ = classmethod(noop_class_getitem)  # type: ignore
+        # models.Manager.__class_getitem__ = classmethod(noop_class_getitem)  # type: ignore
 
         from django.conf import settings
         from django.apps import apps
@@ -119,10 +119,10 @@ class DjangoContext:
             if isinstance(field, Field):
                 yield field
 
-    def get_model_relations(self, model_cls: Type[Model]) -> Iterator[ForeignObjectRel]:
+    def get_model_relations(self, model_cls: Type[Model]) -> Iterator[Tuple[Optional[str], ForeignObjectRel]]:
-        for field in model_cls._meta.get_fields():
+        for relation in model_cls._meta.get_fields():
-            if isinstance(field, ForeignObjectRel):
+            if isinstance(relation, ForeignObjectRel):
-                yield field
+                yield relation.get_accessor_name(), relation
 
     def get_field_lookup_exact_type(self, api: TypeChecker, field: Union[Field, ForeignObjectRel]) -> MypyType:
         if isinstance(field, (RelatedField, ForeignObjectRel)):
@@ -226,7 +226,7 @@ class DjangoContext:
         attname = field.attname
         return attname
 
-    def get_field_nullability(self, field: Union[Field, ForeignObjectRel], method: Optional[str]) -> bool:
+    def get_field_nullability(self, field: Union[Field, ForeignObjectRel], method: Optional[str] = None) -> bool:
         nullable = field.null
         if not nullable and isinstance(field, CharField) and field.blank:
             return True
@@ -356,11 +356,11 @@ class DjangoContext:
                 return AnyType(TypeOfAny.explicit)
 
         if lookup_cls is None or isinstance(lookup_cls, Exact):
-            return self.get_field_lookup_exact_type(helpers.get_typechecker_api(ctx), field)
+            return self.get_field_lookup_exact_type(chk_helpers.get_typechecker_api(ctx), field)
 
         assert lookup_cls is not None
 
-        lookup_info = helpers.lookup_class_typeinfo(helpers.get_typechecker_api(ctx), lookup_cls)
+        lookup_info = helpers.lookup_class_typeinfo(chk_helpers.get_typechecker_api(ctx), lookup_cls)
         if lookup_info is None:
             return AnyType(TypeOfAny.explicit)
 
@@ -370,7 +370,7 @@ class DjangoContext:
                 # if it's Field, consider lookup_type a __get__ of current field
                 if (isinstance(lookup_type, Instance)
                         and lookup_type.type.fullname == fullnames.FIELD_FULLNAME):
-                    field_info = helpers.lookup_class_typeinfo(helpers.get_typechecker_api(ctx), field.__class__)
+                    field_info = helpers.lookup_class_typeinfo(chk_helpers.get_typechecker_api(ctx), field.__class__)
                     if field_info is None:
                         return AnyType(TypeOfAny.explicit)
                     lookup_type = helpers.get_private_descriptor_type(field_info, '_pyi_private_get_type',

mypy_django_plugin/lib/chk_helpers.py

@@ -0,0 +1,120 @@
+from typing import Dict, List, Optional, Set, Union
+
+from mypy import checker
+from mypy.checker import TypeChecker
+from mypy.nodes import (
+    GDEF, MDEF, Expression, MypyFile, SymbolTableNode, TypeInfo, Var,
+)
+from mypy.plugin import (
+    AttributeContext, CheckerPluginInterface, FunctionContext, MethodContext,
+)
+from mypy.types import AnyType, Instance, TupleType
+from mypy.types import Type as MypyType
+from mypy.types import TypedDictType, TypeOfAny
+
+from mypy_django_plugin.lib import helpers
+
+
+def add_new_class_for_current_module(current_module: MypyFile,
+                                     name: str,
+                                     bases: List[Instance],
+                                     fields: Optional[Dict[str, MypyType]] = None
+                                     ) -> TypeInfo:
+    new_class_unique_name = checker.gen_unique_name(name, current_module.names)
+    new_typeinfo = helpers.new_typeinfo(new_class_unique_name,
+                                        bases=bases,
+                                        module_name=current_module.fullname)
+    # new_typeinfo = helpers.make_new_typeinfo_in_current_module(new_class_unique_name,
+    #                                                            bases=bases,
+    #                                                            current_module_fullname=current_module.fullname)
+    # add fields
+    if fields:
+        for field_name, field_type in fields.items():
+            var = Var(field_name, type=field_type)
+            var.info = new_typeinfo
+            var._fullname = new_typeinfo.fullname + '.' + field_name
+            new_typeinfo.names[field_name] = SymbolTableNode(MDEF, var, plugin_generated=True)
+
+    current_module.names[new_class_unique_name] = SymbolTableNode(GDEF, new_typeinfo, plugin_generated=True)
+    current_module.defs.append(new_typeinfo.defn)
+    return new_typeinfo
+
+
+def make_oneoff_named_tuple(api: TypeChecker, name: str, fields: 'Dict[str, MypyType]') -> TupleType:
+    current_module = helpers.get_current_module(api)
+    namedtuple_info = add_new_class_for_current_module(current_module, name,
+                                                       bases=[api.named_generic_type('typing.NamedTuple', [])],
+                                                       fields=fields)
+    return TupleType(list(fields.values()), fallback=Instance(namedtuple_info, []))
+
+
+def make_tuple(api: 'TypeChecker', fields: List[MypyType]) -> TupleType:
+    # fallback for tuples is any builtins.tuple instance
+    fallback = api.named_generic_type('builtins.tuple',
+                                      [AnyType(TypeOfAny.special_form)])
+    return TupleType(fields, fallback=fallback)
+
+
+def make_oneoff_typeddict(api: CheckerPluginInterface, fields: 'Dict[str, MypyType]',
+                          required_keys: Set[str]) -> TypedDictType:
+    object_type = api.named_generic_type('mypy_extensions._TypedDict', [])
+    typed_dict_type = TypedDictType(fields,  # type: ignore
+                                    required_keys=required_keys,
+                                    fallback=object_type)
+    return typed_dict_type
+
+
+def get_typechecker_api(ctx: Union[AttributeContext, MethodContext, FunctionContext]) -> TypeChecker:
+    if not isinstance(ctx.api, TypeChecker):
+        raise ValueError('Not a TypeChecker')
+    return ctx.api
+
+
+def check_types_compatible(ctx: Union[FunctionContext, MethodContext],
+                           *, expected_type: MypyType, actual_type: MypyType, error_message: str) -> None:
+    api = get_typechecker_api(ctx)
+    api.check_subtype(actual_type, expected_type,
+                      ctx.context, error_message,
+                      'got', 'expected')
+
+
+def get_call_argument_by_name(ctx: Union[FunctionContext, MethodContext], name: str) -> Optional[Expression]:
+    """
+    Return the expression for the specific argument.
+    This helper should only be used with non-star arguments.
+    """
+    if name not in ctx.callee_arg_names:
+        return None
+    idx = ctx.callee_arg_names.index(name)
+    args = ctx.args[idx]
+    if len(args) != 1:
+        # Either an error or no value passed.
+        return None
+    return args[0]
+
+
+def get_call_argument_type_by_name(ctx: Union[FunctionContext, MethodContext], name: str) -> Optional[MypyType]:
+    """Return the type for the specific argument.
+
+    This helper should only be used with non-star arguments.
+    """
+    if name not in ctx.callee_arg_names:
+        return None
+    idx = ctx.callee_arg_names.index(name)
+    arg_types = ctx.arg_types[idx]
+    if len(arg_types) != 1:
+        # Either an error or no value passed.
+        return None
+    return arg_types[0]
+
+
+def add_new_sym_for_info(info: TypeInfo, *, name: str, sym_type: MypyType) -> None:
+    # type=: type of the variable itself
+    var = Var(name=name, type=sym_type)
+    # var.info: type of the object variable is bound to
+    var.info = info
+    var._fullname = info.fullname + '.' + name
+    var.is_initialized_in_class = True
+    var.is_inferred = True
+    info.names[name] = SymbolTableNode(MDEF, var,
+                                       plugin_generated=True)

mypy_django_plugin/lib/generics.py

@@ -0,0 +1,6 @@
+def make_classes_generic(*klasses: type) -> None:
+    for klass in klasses:
+        def fake_classgetitem(cls, *args, **kwargs):
+            return cls
+
+        klass.__class_getitem__ = classmethod(fake_classgetitem)  # type: ignore

mypy_django_plugin/lib/helpers.py

@@ -1,70 +1,372 @@
-from collections import OrderedDict
+from abc import abstractmethod
 from typing import (
-    TYPE_CHECKING, Any, Dict, Iterable, Iterator, List, Optional, Set, Tuple, Union,
+    TYPE_CHECKING, Any, Dict, Iterable, Iterator, List, Optional, Tuple, Union,
-)
+    cast)
 
-from django.db.models.fields import Field
 from django.db.models.fields.related import RelatedField
 from django.db.models.fields.reverse_related import ForeignObjectRel
-from mypy import checker
 from mypy.checker import TypeChecker
 from mypy.mro import calculate_mro
 from mypy.nodes import (
-    GDEF, MDEF, Argument, Block, ClassDef, Expression, FuncDef, MemberExpr, MypyFile, NameExpr, PlaceholderNode,
+    Block, ClassDef, Expression, MemberExpr, MypyFile, NameExpr, StrExpr, SymbolTable, SymbolTableNode,
-    StrExpr, SymbolNode, SymbolTable, SymbolTableNode, TypeInfo, Var,
+    TypeInfo, Var,
-)
+    CallExpr, Context, PlaceholderNode, FuncDef, FakeInfo, OverloadedFuncDef, Decorator)
-from mypy.plugin import (
+from mypy.plugin import DynamicClassDefContext, ClassDefContext, AttributeContext, MethodContext
-    AttributeContext, CheckerPluginInterface, ClassDefContext, DynamicClassDefContext, FunctionContext, MethodContext,
-)
 from mypy.plugins.common import add_method
-from mypy.semanal import SemanticAnalyzer
+from mypy.semanal import SemanticAnalyzer, is_valid_replacement, is_same_symbol
-from mypy.types import AnyType, CallableType, Instance, NoneTyp, TupleType
+from mypy.types import AnyType, Instance, NoneTyp, TypeType, ProperType, CallableType
 from mypy.types import Type as MypyType
-from mypy.types import TypedDictType, TypeOfAny, UnionType
+from mypy.types import TypeOfAny, UnionType
+from mypy.typetraverser import TypeTraverserVisitor
 
+from django.db.models.fields import Field
 from mypy_django_plugin.lib import fullnames
+from mypy_django_plugin.lib.sem_helpers import prepare_unannotated_method_signature, analyze_callable_signature
+from mypy_django_plugin.transformers2 import new_helpers
 
 if TYPE_CHECKING:
     from mypy_django_plugin.django.context import DjangoContext
+    from mypy_django_plugin.main import NewSemanalDjangoPlugin
+
+AnyPluginAPI = Union[TypeChecker, SemanticAnalyzer]
+
+
+class DjangoPluginCallback:
+    django_context: 'DjangoContext'
+
+    def __init__(self, plugin: 'NewSemanalDjangoPlugin') -> None:
+        self.plugin = plugin
+        self.django_context = plugin.django_context
+
+    def new_typeinfo(self, name: str, bases: List[Instance]) -> TypeInfo:
+        class_def = ClassDef(name, Block([]))
+        class_def.fullname = self.qualified_name(name)
+
+        info = TypeInfo(SymbolTable(), class_def, self.get_current_module().fullname)
+        info.bases = bases
+        calculate_mro(info)
+        info.metaclass_type = info.calculate_metaclass_type()
+
+        class_def.info = info
+        return info
+
+    @abstractmethod
+    def get_current_module(self) -> MypyFile:
+        raise NotImplementedError()
+
+    @abstractmethod
+    def qualified_name(self, name: str) -> str:
+        raise NotImplementedError()
+
+
+class SemanalPluginCallback(DjangoPluginCallback):
+    semanal_api: SemanticAnalyzer
+
+    def build_defer_error_message(self, message: str) -> str:
+        return f'{self.__class__.__name__}: {message}'
+
+    def defer_till_next_iteration(self, deferral_context: Optional[Context] = None,
+                                  *,
+                                  reason: Optional[str] = None) -> bool:
+        """ Returns False if cannot be deferred. """
+        if self.semanal_api.final_iteration:
+            return False
+        self.semanal_api.defer(deferral_context)
+        print(f'LOG: defer: {self.build_defer_error_message(reason)}')
+        return True
+
+    def get_current_module(self) -> MypyFile:
+        return self.semanal_api.cur_mod_node
+
+    def qualified_name(self, name: str) -> str:
+        return self.semanal_api.qualified_name(name)
+
+    def lookup_typeinfo_or_defer(self, fullname: str, *,
+                                 deferral_context: Optional[Context] = None,
+                                 reason_for_defer: Optional[str] = None) -> Optional[TypeInfo]:
+        sym = self.plugin.lookup_fully_qualified(fullname)
+        if sym is None or sym.node is None or isinstance(sym.node, PlaceholderNode):
+            deferral_context = deferral_context or self.semanal_api.cur_mod_node
+            reason = reason_for_defer or f'{fullname!r} is not available for lookup'
+            if not self.defer_till_next_iteration(deferral_context, reason=reason):
+                raise new_helpers.TypeInfoNotFound(fullname)
+            return None
+
+        if not isinstance(sym.node, TypeInfo):
+            raise ValueError(f'{fullname!r} does not correspond to TypeInfo')
+
+        return sym.node
+
+    def new_typeinfo(self, name: str, bases: List[Instance], module_fullname: Optional[str] = None) -> TypeInfo:
+        class_def = ClassDef(name, Block([]))
+        class_def.fullname = self.semanal_api.qualified_name(name)
+
+        info = TypeInfo(SymbolTable(), class_def,
+                        module_fullname or self.get_current_module().fullname)
+        info.bases = bases
+        calculate_mro(info)
+        info.metaclass_type = info.calculate_metaclass_type()
+
+        class_def.info = info
+        return info
+
+    def add_symbol_table_node(self,
+                              name: str,
+                              symbol: SymbolTableNode,
+                              symbol_table: Optional[SymbolTable] = None,
+                              context: Optional[Context] = None,
+                              can_defer: bool = True,
+                              escape_comprehensions: bool = False) -> None:
+        """ Patched copy of SemanticAnalyzer.add_symbol_table_node(). """
+        names = symbol_table or self.semanal_api.current_symbol_table(escape_comprehensions=escape_comprehensions)
+        existing = names.get(name)
+        if isinstance(symbol.node, PlaceholderNode) and can_defer:
+            self.semanal_api.defer(context)
+            return None
+        if (existing is not None
+                and context is not None
+                and not is_valid_replacement(existing, symbol)):
+            # There is an existing node, so this may be a redefinition.
+            # If the new node points to the same node as the old one,
+            # or if both old and new nodes are placeholders, we don't
+            # need to do anything.
+            old = existing.node
+            new = symbol.node
+            if isinstance(new, PlaceholderNode):
+                # We don't know whether this is okay. Let's wait until the next iteration.
+                return False
+            if not is_same_symbol(old, new):
+                if isinstance(new, (FuncDef, Decorator, OverloadedFuncDef, TypeInfo)):
+                    self.semanal_api.add_redefinition(names, name, symbol)
+                if not (isinstance(new, (FuncDef, Decorator))
+                        and self.semanal_api.set_original_def(old, new)):
+                    self.semanal_api.name_already_defined(name, context, existing)
+        elif name not in self.semanal_api.missing_names and '*' not in self.semanal_api.missing_names:
+            names[name] = symbol
+            self.progress = True
+            return None
+        raise new_helpers.SymbolAdditionNotPossible()
+
+    # def add_symbol_table_node_or_defer(self, name: str, sym: SymbolTableNode) -> bool:
+    #     return self.semanal_api.add_symbol_table_node(name, sym,
+    #                                                   context=self.semanal_api.cur_mod_node)
+
+    def add_method_from_signature(self,
+                                  signature_node: FuncDef,
+                                  new_method_name: str,
+                                  new_self_type: Instance,
+                                  class_defn: ClassDef) -> bool:
+        if signature_node.type is None:
+            if self.defer_till_next_iteration(reason=signature_node.fullname):
+                return False
+
+            arguments, return_type = prepare_unannotated_method_signature(signature_node)
+            ctx = ClassDefContext(class_defn, signature_node, self.semanal_api)
+            add_method(ctx,
+                       new_method_name,
+                       self_type=new_self_type,
+                       args=arguments,
+                       return_type=return_type)
+            return True
+
+        # add imported objects from method signature to the current module, if not present
+        source_symbols = self.semanal_api.modules[signature_node.info.module_name].names
+        currently_imported_symbols = self.semanal_api.cur_mod_node.names
+
+        def import_symbol_from_source(name: str) -> None:
+            if name in source_symbols['__builtins__'].node.names:
+                return
+            sym = source_symbols[name].copy()
+            self.semanal_api.add_imported_symbol(name, sym, context=self.semanal_api.cur_mod_node)
+
+        class UnimportedTypesVisitor(TypeTraverserVisitor):
+            def visit_instance(self, t: Instance) -> None:
+                super().visit_instance(t)
+                if isinstance(t.type, FakeInfo):
+                    return
+                type_name = t.type.name
+                sym = currently_imported_symbols.get(type_name)
+                if sym is None:
+                    import_symbol_from_source(type_name)
+
+        signature_node.type.accept(UnimportedTypesVisitor())
+
+        # # copy global SymbolTableNode objects from original class to the current node, if not present
+        # original_module = semanal_api.modules[method_node.info.module_name]
+        # for name, sym in original_module.names.items():
+        #     if (not sym.plugin_generated
+        #             and name not in semanal_api.cur_mod_node.names):
+        #         semanal_api.add_imported_symbol(name, sym, context=semanal_api.cur_mod_node)
+
+        arguments, analyzed_return_type, unbound = analyze_callable_signature(self.semanal_api, signature_node)
+        if unbound:
+            raise new_helpers.IncompleteDefnError(f'Signature of method {signature_node.fullname!r} is not ready')
+
+        assert len(arguments) + 1 == len(signature_node.arguments)
+        assert analyzed_return_type is not None
+
+        ctx = ClassDefContext(class_defn, signature_node, self.semanal_api)
+        add_method(ctx,
+                   new_method_name,
+                   self_type=new_self_type,
+                   args=arguments,
+                   return_type=analyzed_return_type)
+        return True
+
+
+class DynamicClassPluginCallback(SemanalPluginCallback):
+    class_name: str
+    call_expr: CallExpr
+
+    def __call__(self, ctx: DynamicClassDefContext) -> None:
+        self.class_name = ctx.name
+        self.call_expr = ctx.call
+        self.semanal_api = cast(SemanticAnalyzer, ctx.api)
+        self.create_new_dynamic_class()
+
+    def get_callee(self) -> MemberExpr:
+        callee = self.call_expr.callee
+        assert isinstance(callee, MemberExpr)
+        return callee
+
+    def lookup_same_module_or_defer(self, name: str, *,
+                                    deferral_context: Optional[Context] = None) -> Optional[SymbolTableNode]:
+        sym = self.semanal_api.lookup_qualified(name, self.call_expr)
+        if sym is None or sym.node is None or isinstance(sym.node, PlaceholderNode):
+            deferral_context = deferral_context or self.call_expr
+            if not self.defer_till_next_iteration(deferral_context,
+                                                  reason=f'{self.semanal_api.cur_mod_id}.{name} does not exist'):
+                raise new_helpers.NameNotFound(name)
+            return None
+        return sym
+
+    @abstractmethod
+    def create_new_dynamic_class(self) -> None:
+        raise NotImplementedError
+
+
+class ClassDefPluginCallback(SemanalPluginCallback):
+    reason: Expression
+    class_defn: ClassDef
+    ctx: ClassDefContext
+
+    def __call__(self, ctx: ClassDefContext) -> None:
+        self.reason = ctx.reason
+        self.class_defn = ctx.cls
+        self.semanal_api = cast(SemanticAnalyzer, ctx.api)
+        self.ctx = ctx
+        self.modify_class_defn()
+
+    @abstractmethod
+    def modify_class_defn(self) -> None:
+        raise NotImplementedError
+
+
+class TypeCheckerPluginCallback(DjangoPluginCallback):
+    type_checker: TypeChecker
+
+    def get_current_module(self) -> MypyFile:
+        current_module = None
+        for item in reversed(self.type_checker.scope.stack):
+            if isinstance(item, MypyFile):
+                current_module = item
+                break
+        assert current_module is not None
+        return current_module
+
+    def qualified_name(self, name: str) -> str:
+        return self.type_checker.scope.stack[-1].fullname + '.' + name
+
+    def lookup_typeinfo(self, fullname: str) -> Optional[TypeInfo]:
+        sym = self.plugin.lookup_fully_qualified(fullname)
+        if sym is None or sym.node is None:
+            return None
+        if not isinstance(sym.node, TypeInfo):
+            raise ValueError(f'{fullname!r} does not correspond to TypeInfo')
+        return sym.node
+
+
+class GetMethodPluginCallback(TypeCheckerPluginCallback):
+    callee_type: Instance
+    ctx: MethodContext
+
+    def __call__(self, ctx: MethodContext) -> MypyType:
+        self.type_checker = ctx.api
+
+        assert isinstance(ctx.type, CallableType)
+        self.callee_type = ctx.type.ret_type
+        self.ctx = ctx
+        return self.get_method_return_type()
+
+    @abstractmethod
+    def get_method_return_type(self) -> MypyType:
+        raise NotImplementedError
+
+
+class GetAttributeCallback(TypeCheckerPluginCallback):
+    obj_type: ProperType
+    default_attr_type: MypyType
+    error_context: MemberExpr
+    name: str
+
+    def __call__(self, ctx: AttributeContext) -> MypyType:
+        self.ctx = ctx
+        self.type_checker = ctx.api
+        self.obj_type = ctx.type
+        self.default_attr_type = ctx.default_attr_type
+        self.error_context = ctx.context
+        assert isinstance(self.error_context, MemberExpr)
+        self.name = self.error_context.name
+        return self.default_attr_type
 
 
 def get_django_metadata(model_info: TypeInfo) -> Dict[str, Any]:
     return model_info.metadata.setdefault('django', {})
 
 
-class IncompleteDefnException(Exception):
+def split_symbol_name(fullname: str, all_modules: Dict[str, MypyFile]) -> Optional[Tuple[str, str]]:
-    pass
-
-
-def lookup_fully_qualified_sym(fullname: str, all_modules: Dict[str, MypyFile]) -> Optional[SymbolTableNode]:
     if '.' not in fullname:
         return None
-    module, cls_name = fullname.rsplit('.', 1)
+    module_name = None
-
+    parts = fullname.split('.')
-    module_file = all_modules.get(module)
+    for i in range(len(parts), 0, -1):
-    if module_file is None:
+        possible_module_name = '.'.join(parts[:i])
+        if possible_module_name in all_modules:
+            module_name = possible_module_name
+            break
+    if module_name is None:
         return None
-    sym = module_file.names.get(cls_name)
+
-    if sym is None:
+    symbol_name = fullname.replace(module_name, '').lstrip('.')
+    return module_name, symbol_name
+
+
+def lookup_fully_qualified_typeinfo(api: AnyPluginAPI, fullname: str) -> Optional[TypeInfo]:
+    split = split_symbol_name(fullname, api.modules)
+    if split is None:
         return None
-    return sym
+    module_name, cls_name = split
 
+    sym_table = api.modules[module_name].names  # type: Dict[str, SymbolTableNode]
+    if '.' in cls_name:
+        parent_cls_name, _, cls_name = cls_name.rpartition('.')
+        # nested class
+        for parent_cls_name in parent_cls_name.split('.'):
+            sym = sym_table.get(parent_cls_name)
+            if (sym is None or sym.node is None
+                    or not isinstance(sym.node, TypeInfo)):
+                return None
+            sym_table = sym.node.names
 
-def lookup_fully_qualified_generic(name: str, all_modules: Dict[str, MypyFile]) -> Optional[SymbolNode]:
+    sym = sym_table.get(cls_name)
-    sym = lookup_fully_qualified_sym(name, all_modules)
+    if (sym is None
-    if sym is None:
+            or sym.node is None
+            or not isinstance(sym.node, TypeInfo)):
         return None
     return sym.node
 
 
-def lookup_fully_qualified_typeinfo(api: Union[TypeChecker, SemanticAnalyzer], fullname: str) -> Optional[TypeInfo]:
+def lookup_class_typeinfo(api: AnyPluginAPI, klass: type) -> Optional[TypeInfo]:
-    node = lookup_fully_qualified_generic(fullname, api.modules)
-    if not isinstance(node, TypeInfo):
-        return None
-    return node
-
-
-def lookup_class_typeinfo(api: TypeChecker, klass: type) -> Optional[TypeInfo]:
     fullname = get_class_fullname(klass)
     field_info = lookup_fully_qualified_typeinfo(api, fullname)
     return field_info
@@ -79,36 +381,6 @@ def get_class_fullname(klass: type) -> str:
     return klass.__module__ + '.' + klass.__qualname__
 
 
-def get_call_argument_by_name(ctx: Union[FunctionContext, MethodContext], name: str) -> Optional[Expression]:
-    """
-    Return the expression for the specific argument.
-    This helper should only be used with non-star arguments.
-    """
-    if name not in ctx.callee_arg_names:
-        return None
-    idx = ctx.callee_arg_names.index(name)
-    args = ctx.args[idx]
-    if len(args) != 1:
-        # Either an error or no value passed.
-        return None
-    return args[0]
-
-
-def get_call_argument_type_by_name(ctx: Union[FunctionContext, MethodContext], name: str) -> Optional[MypyType]:
-    """Return the type for the specific argument.
-
-    This helper should only be used with non-star arguments.
-    """
-    if name not in ctx.callee_arg_names:
-        return None
-    idx = ctx.callee_arg_names.index(name)
-    arg_types = ctx.arg_types[idx]
-    if len(arg_types) != 1:
-        # Either an error or no value passed.
-        return None
-    return arg_types[0]
-
-
 def make_optional(typ: MypyType) -> MypyType:
     return UnionType.make_union([typ, NoneTyp()])
 
@@ -153,7 +425,7 @@ def get_private_descriptor_type(type_info: TypeInfo, private_field_name: str, is
     return AnyType(TypeOfAny.explicit)
 
 
-def get_field_lookup_exact_type(api: TypeChecker, field: Field) -> MypyType:
+def get_field_lookup_exact_type(api: AnyPluginAPI, field: Field) -> MypyType:
     if isinstance(field, (RelatedField, ForeignObjectRel)):
         lookup_type_class = field.related_model
         rel_model_info = lookup_class_typeinfo(api, lookup_type_class)
@@ -168,44 +440,10 @@ def get_field_lookup_exact_type(api: TypeChecker, field: Field) -> MypyType:
                                        is_nullable=field.null)
 
 
-def get_nested_meta_node_for_current_class(info: TypeInfo) -> Optional[TypeInfo]:
+def get_current_module(api: AnyPluginAPI) -> MypyFile:
-    metaclass_sym = info.names.get('Meta')
+    if isinstance(api, SemanticAnalyzer):
-    if metaclass_sym is not None and isinstance(metaclass_sym.node, TypeInfo):
+        return api.cur_mod_node
-        return metaclass_sym.node
-    return None
 
-
-def add_new_class_for_module(module: MypyFile,
-                             name: str,
-                             bases: List[Instance],
-                             fields: Optional[Dict[str, MypyType]] = None
-                             ) -> TypeInfo:
-    new_class_unique_name = checker.gen_unique_name(name, module.names)
-
-    # make new class expression
-    classdef = ClassDef(new_class_unique_name, Block([]))
-    classdef.fullname = module.fullname + '.' + new_class_unique_name
-
-    # make new TypeInfo
-    new_typeinfo = TypeInfo(SymbolTable(), classdef, module.fullname)
-    new_typeinfo.bases = bases
-    calculate_mro(new_typeinfo)
-    new_typeinfo.calculate_metaclass_type()
-
-    # add fields
-    if fields:
-        for field_name, field_type in fields.items():
-            var = Var(field_name, type=field_type)
-            var.info = new_typeinfo
-            var._fullname = new_typeinfo.fullname + '.' + field_name
-            new_typeinfo.names[field_name] = SymbolTableNode(MDEF, var, plugin_generated=True)
-
-    classdef.info = new_typeinfo
-    module.names[new_class_unique_name] = SymbolTableNode(GDEF, new_typeinfo, plugin_generated=True)
-    return new_typeinfo
-
-
-def get_current_module(api: TypeChecker) -> MypyFile:
     current_module = None
     for item in reversed(api.scope.stack):
         if isinstance(item, MypyFile):
@@ -215,21 +453,6 @@ def get_current_module(api: TypeChecker) -> MypyFile:
     return current_module
 
 
-def make_oneoff_named_tuple(api: TypeChecker, name: str, fields: 'OrderedDict[str, MypyType]') -> TupleType:
-    current_module = get_current_module(api)
-    namedtuple_info = add_new_class_for_module(current_module, name,
-                                               bases=[api.named_generic_type('typing.NamedTuple', [])],
-                                               fields=fields)
-    return TupleType(list(fields.values()), fallback=Instance(namedtuple_info, []))
-
-
-def make_tuple(api: 'TypeChecker', fields: List[MypyType]) -> TupleType:
-    # fallback for tuples is any builtins.tuple instance
-    fallback = api.named_generic_type('builtins.tuple',
-                                      [AnyType(TypeOfAny.special_form)])
-    return TupleType(fields, fallback=fallback)
-
-
 def convert_any_to_type(typ: MypyType, referred_to_type: MypyType) -> MypyType:
     if isinstance(typ, UnionType):
         converted_items = []
@@ -252,13 +475,6 @@ def convert_any_to_type(typ: MypyType, referred_to_type: MypyType) -> MypyType:
     return typ
 
 
-def make_typeddict(api: CheckerPluginInterface, fields: 'OrderedDict[str, MypyType]',
-                   required_keys: Set[str]) -> TypedDictType:
-    object_type = api.named_generic_type('mypy_extensions._TypedDict', [])
-    typed_dict_type = TypedDictType(fields, required_keys=required_keys, fallback=object_type)
-    return typed_dict_type
-
-
 def resolve_string_attribute_value(attr_expr: Expression, django_context: 'DjangoContext') -> Optional[str]:
     if isinstance(attr_expr, StrExpr):
         return attr_expr.value
@@ -272,104 +488,25 @@ def resolve_string_attribute_value(attr_expr: Expression, django_context: 'Djang
     return None
 
 
-def get_semanal_api(ctx: Union[ClassDefContext, DynamicClassDefContext]) -> SemanticAnalyzer:
+def is_subclass_of_model(info: TypeInfo, django_context: 'DjangoContext') -> bool:
-    if not isinstance(ctx.api, SemanticAnalyzer):
-        raise ValueError('Not a SemanticAnalyzer')
-    return ctx.api
-
-
-def get_typechecker_api(ctx: Union[AttributeContext, MethodContext, FunctionContext]) -> TypeChecker:
-    if not isinstance(ctx.api, TypeChecker):
-        raise ValueError('Not a TypeChecker')
-    return ctx.api
-
-
-def is_model_subclass_info(info: TypeInfo, django_context: 'DjangoContext') -> bool:
     return (info.fullname in django_context.all_registered_model_class_fullnames
             or info.has_base(fullnames.MODEL_CLASS_FULLNAME))
 
 
-def check_types_compatible(ctx: Union[FunctionContext, MethodContext],
+def new_typeinfo(name: str,
-                           *, expected_type: MypyType, actual_type: MypyType, error_message: str) -> None:
+                 *,
-    api = get_typechecker_api(ctx)
+                 bases: List[Instance],
-    api.check_subtype(actual_type, expected_type,
+                 module_name: str) -> TypeInfo:
-                      ctx.context, error_message,
+    """
-                      'got', 'expected')
+        Construct new TypeInfo instance. Cannot be used for nested classes.
+    """
+    class_def = ClassDef(name, Block([]))
+    class_def.fullname = module_name + '.' + name
 
+    info = TypeInfo(SymbolTable(), class_def, module_name)
+    info.bases = bases
+    calculate_mro(info)
+    info.metaclass_type = info.calculate_metaclass_type()
 
-def add_new_sym_for_info(info: TypeInfo, *, name: str, sym_type: MypyType) -> None:
+    class_def.info = info
-    # type=: type of the variable itself
+    return info
-    var = Var(name=name, type=sym_type)
-    # var.info: type of the object variable is bound to
-    var.info = info
-    var._fullname = info.fullname + '.' + name
-    var.is_initialized_in_class = True
-    var.is_inferred = True
-    info.names[name] = SymbolTableNode(MDEF, var,
-                                       plugin_generated=True)
-
-
-def build_unannotated_method_args(method_node: FuncDef) -> Tuple[List[Argument], MypyType]:
-    prepared_arguments = []
-    for argument in method_node.arguments[1:]:
-        argument.type_annotation = AnyType(TypeOfAny.unannotated)
-        prepared_arguments.append(argument)
-    return_type = AnyType(TypeOfAny.unannotated)
-    return prepared_arguments, return_type
-
-
-def copy_method_to_another_class(ctx: ClassDefContext, self_type: Instance,
-                                 new_method_name: str, method_node: FuncDef) -> None:
-    semanal_api = get_semanal_api(ctx)
-    if method_node.type is None:
-        if not semanal_api.final_iteration:
-            semanal_api.defer()
-            return
-
-        arguments, return_type = build_unannotated_method_args(method_node)
-        add_method(ctx,
-                   new_method_name,
-                   args=arguments,
-                   return_type=return_type,
-                   self_type=self_type)
-        return
-
-    method_type = method_node.type
-    if not isinstance(method_type, CallableType):
-        if not semanal_api.final_iteration:
-            semanal_api.defer()
-        return
-
-    arguments = []
-    bound_return_type = semanal_api.anal_type(method_type.ret_type,
-                                              allow_placeholder=True)
-    assert bound_return_type is not None
-
-    if isinstance(bound_return_type, PlaceholderNode):
-        return
-
-    for arg_name, arg_type, original_argument in zip(method_type.arg_names[1:],
-                                                     method_type.arg_types[1:],
-                                                     method_node.arguments[1:]):
-        bound_arg_type = semanal_api.anal_type(arg_type, allow_placeholder=True)
-        assert bound_arg_type is not None
-
-        if isinstance(bound_arg_type, PlaceholderNode):
-            return
-
-        var = Var(name=original_argument.variable.name,
-                  type=arg_type)
-        var.line = original_argument.variable.line
-        var.column = original_argument.variable.column
-        argument = Argument(variable=var,
-                            type_annotation=bound_arg_type,
-                            initializer=original_argument.initializer,
-                            kind=original_argument.kind)
-        argument.set_line(original_argument)
-        arguments.append(argument)
-
-    add_method(ctx,
-               new_method_name,
-               args=arguments,
-               return_type=bound_return_type,
-               self_type=self_type)

mypy_django_plugin/lib/sem_helpers.py

@@ -0,0 +1,120 @@
+from typing import List, NamedTuple, Optional, Tuple, Union, cast
+
+from mypy.nodes import Argument, FuncDef, TypeInfo, Var
+from mypy.plugin import ClassDefContext, DynamicClassDefContext
+from mypy.plugins.common import add_method
+from mypy.semanal import SemanticAnalyzer
+from mypy.types import AnyType, CallableType, Instance, PlaceholderType
+from mypy.types import Type as MypyType
+from mypy.types import TypeOfAny, get_proper_type
+
+
+class IncompleteDefnError(Exception):
+    def __init__(self, error_message: str = '') -> None:
+        super().__init__(error_message)
+
+
+class BoundNameNotFound(IncompleteDefnError):
+    def __init__(self, fullname: str) -> None:
+        super().__init__(f'No {fullname!r} found')
+
+
+def get_semanal_api(ctx: Union[ClassDefContext, DynamicClassDefContext]) -> SemanticAnalyzer:
+    return cast(SemanticAnalyzer, ctx.api)
+
+
+def get_nested_meta_node_for_current_class(info: TypeInfo) -> Optional[TypeInfo]:
+    metaclass_sym = info.names.get('Meta')
+    if metaclass_sym is not None and isinstance(metaclass_sym.node, TypeInfo):
+        return metaclass_sym.node
+    return None
+
+
+def prepare_unannotated_method_signature(method_node: FuncDef) -> Tuple[List[Argument], MypyType]:
+    prepared_arguments = []
+    for argument in method_node.arguments[1:]:
+        argument.type_annotation = AnyType(TypeOfAny.unannotated)
+        prepared_arguments.append(argument)
+    return_type = AnyType(TypeOfAny.unannotated)
+    return prepared_arguments, return_type
+
+
+class SignatureTuple(NamedTuple):
+    arguments: List[Argument]
+    return_type: Optional[MypyType]
+    cannot_be_bound: bool
+
+
+def analyze_callable_signature(api: SemanticAnalyzer, method_node: FuncDef) -> SignatureTuple:
+    method_type = method_node.type
+    assert isinstance(method_type, CallableType)
+
+    arguments = []
+    unbound = False
+    for arg_name, arg_type, original_argument in zip(method_type.arg_names[1:],
+                                                     method_type.arg_types[1:],
+                                                     method_node.arguments[1:]):
+        analyzed_arg_type = api.anal_type(get_proper_type(arg_type), allow_placeholder=True)
+        assert analyzed_arg_type is not None
+        if isinstance(analyzed_arg_type, PlaceholderType):
+            unbound = True
+
+        var = Var(name=original_argument.variable.name,
+                  type=analyzed_arg_type)
+        var.set_line(original_argument.variable)
+
+        argument = Argument(variable=var,
+                            type_annotation=analyzed_arg_type,
+                            initializer=original_argument.initializer,
+                            kind=original_argument.kind)
+        argument.set_line(original_argument)
+        arguments.append(argument)
+
+    analyzed_ret_type = api.anal_type(get_proper_type(method_type.ret_type), allow_placeholder=True)
+    assert analyzed_ret_type is not None
+    if isinstance(analyzed_ret_type, PlaceholderType):
+        unbound = True
+    return SignatureTuple(arguments, analyzed_ret_type, unbound)
+
+
+def copy_method_or_incomplete_defn_exception(ctx: ClassDefContext,
+                                             self_type: Instance,
+                                             new_method_name: str,
+                                             method_node: FuncDef) -> None:
+    semanal_api = get_semanal_api(ctx)
+
+    if method_node.type is None:
+        if not semanal_api.final_iteration:
+            raise IncompleteDefnError(f'Unannotated method {method_node.fullname!r}')
+
+        arguments, return_type = prepare_unannotated_method_signature(method_node)
+        add_method(ctx,
+                   new_method_name,
+                   args=arguments,
+                   return_type=return_type,
+                   self_type=self_type)
+        return
+
+    assert isinstance(method_node.type, CallableType)
+
+    # copy global SymbolTableNode objects from original class to the current node, if not present
+    original_module = semanal_api.modules[method_node.info.module_name]
+    for name, sym in original_module.names.items():
+        if (not sym.plugin_generated
+                and name not in semanal_api.cur_mod_node.names):
+            semanal_api.add_imported_symbol(name, sym, context=semanal_api.cur_mod_node)
+
+    arguments, analyzed_return_type, unbound = analyze_callable_signature(semanal_api, method_node)
+    assert len(arguments) + 1 == len(method_node.arguments)
+    if unbound:
+        raise IncompleteDefnError(f'Signature of method {method_node.fullname!r} is not ready')
+
+    assert analyzed_return_type is not None
+
+    if new_method_name in ctx.cls.info.names:
+        del ctx.cls.info.names[new_method_name]
+    add_method(ctx,
+               new_method_name,
+               args=arguments,
+               return_type=analyzed_return_type,
+               self_type=self_type)

mypy_django_plugin/main.py

@@ -17,10 +17,10 @@ from mypy_django_plugin.lib import fullnames, helpers
 from mypy_django_plugin.transformers import (
     fields, forms, init_create, meta, querysets, request, settings,
 )
-from mypy_django_plugin.transformers.managers import (
-    create_new_manager_class_from_from_queryset_method,
-)
 from mypy_django_plugin.transformers.models import process_model_class
+from mypy_django_plugin.transformers2.dynamic_managers import CreateNewManagerClassFrom_FromQuerySet
+from mypy_django_plugin.transformers2.models import ModelCallback
+from mypy_django_plugin.transformers2.related_managers import GetRelatedManagerCallback
 
 
 def transform_model_class(ctx: ClassDefContext,
@@ -123,6 +123,10 @@ class NewSemanalDjangoPlugin(Plugin):
         return 10, module, -1
 
     def get_additional_deps(self, file: MypyFile) -> List[Tuple[int, str, int]]:
+        # load QuerySet and Manager together (for as_manager)
+        if file.fullname == 'django.db.models.query':
+            return [self._new_dependency('django.db.models.manager')]
+
         # for settings
         if file.fullname == 'django.conf' and self.django_context.django_settings_module:
             return [self._new_dependency(self.django_context.django_settings_module)]
@@ -171,16 +175,12 @@ class NewSemanalDjangoPlugin(Plugin):
         if fullname == 'django.contrib.auth.get_user_model':
             return partial(settings.get_user_model_hook, django_context=self.django_context)
 
-        manager_bases = self._get_current_manager_bases()
-        if fullname in manager_bases:
-            return querysets.determine_proper_manager_type
-
         info = self._get_typeinfo_or_none(fullname)
         if info:
             if info.has_base(fullnames.FIELD_FULLNAME):
                 return partial(fields.transform_into_proper_return_type, django_context=self.django_context)
 
-            if helpers.is_model_subclass_info(info, self.django_context):
+            if helpers.is_subclass_of_model(info, self.django_context):
                 return partial(init_create.redefine_and_typecheck_model_init, django_context=self.django_context)
         return None
 
@@ -224,7 +224,7 @@ class NewSemanalDjangoPlugin(Plugin):
                             ) -> Optional[Callable[[ClassDefContext], None]]:
         if (fullname in self.django_context.all_registered_model_class_fullnames
                 or fullname in self._get_current_model_bases()):
-            return partial(transform_model_class, django_context=self.django_context)
+            return ModelCallback(self)
 
         if fullname in self._get_current_manager_bases():
             return add_new_manager_base
@@ -243,6 +243,10 @@ class NewSemanalDjangoPlugin(Plugin):
         info = self._get_typeinfo_or_none(class_name)
         if info and info.has_base(fullnames.HTTPREQUEST_CLASS_FULLNAME) and attr_name == 'user':
             return partial(request.set_auth_user_model_as_type_for_request_user, django_context=self.django_context)
+
+        if info and info.has_base(fullnames.MODEL_CLASS_FULLNAME):
+            return GetRelatedManagerCallback(self)
+
         return None
 
     def get_dynamic_class_hook(self, fullname: str
@@ -251,7 +255,8 @@ class NewSemanalDjangoPlugin(Plugin):
             class_name, _, _ = fullname.rpartition('.')
             info = self._get_typeinfo_or_none(class_name)
             if info and info.has_base(fullnames.BASE_MANAGER_CLASS_FULLNAME):
-                return create_new_manager_class_from_from_queryset_method
+                return CreateNewManagerClassFrom_FromQuerySet(self)
+
         return None
 
 

mypy_django_plugin/transformers/fields.py

@@ -9,13 +9,13 @@ from mypy.types import Type as MypyType
 from mypy.types import TypeOfAny
 
 from mypy_django_plugin.django.context import DjangoContext
-from mypy_django_plugin.lib import fullnames, helpers
+from mypy_django_plugin.lib import chk_helpers, fullnames, helpers
 
 
 def _get_current_field_from_assignment(ctx: FunctionContext, django_context: DjangoContext) -> Optional[Field]:
-    outer_model_info = helpers.get_typechecker_api(ctx).scope.active_class()
+    outer_model_info = chk_helpers.get_typechecker_api(ctx).scope.active_class()
     if (outer_model_info is None
-            or not helpers.is_model_subclass_info(outer_model_info, django_context)):
+            or not helpers.is_subclass_of_model(outer_model_info, django_context)):
         return None
 
     field_name = None
@@ -66,12 +66,12 @@ def fill_descriptor_types_for_related_field(ctx: FunctionContext, django_context
         # __get__/__set__ of ForeignKey of derived model
         for model_cls in django_context.all_registered_model_classes:
             if issubclass(model_cls, current_model_cls) and not model_cls._meta.abstract:
-                derived_model_info = helpers.lookup_class_typeinfo(helpers.get_typechecker_api(ctx), model_cls)
+                derived_model_info = helpers.lookup_class_typeinfo(chk_helpers.get_typechecker_api(ctx), model_cls)
                 if derived_model_info is not None:
                     fk_ref_type = Instance(derived_model_info, [])
                     derived_fk_type = reparametrize_related_field_type(default_related_field_type,
                                                                        set_type=fk_ref_type, get_type=fk_ref_type)
-                    helpers.add_new_sym_for_info(derived_model_info,
+                    chk_helpers.add_new_sym_for_info(derived_model_info,
                                                      name=current_field.name,
                                                      sym_type=derived_fk_type)
 
@@ -80,7 +80,7 @@ def fill_descriptor_types_for_related_field(ctx: FunctionContext, django_context
     if related_model_to_set._meta.proxy_for_model is not None:
         related_model_to_set = related_model_to_set._meta.proxy_for_model
 
-    typechecker_api = helpers.get_typechecker_api(ctx)
+    typechecker_api = chk_helpers.get_typechecker_api(ctx)
 
     related_model_info = helpers.lookup_class_typeinfo(typechecker_api, related_model)
     if related_model_info is None:
@@ -110,11 +110,16 @@ def get_field_descriptor_types(field_info: TypeInfo, is_nullable: bool) -> Tuple
     return set_type, get_type
 
 
+def get_field_type(field_info: TypeInfo, is_nullable: bool) -> Instance:
+    set_type, get_type = get_field_descriptor_types(field_info, is_nullable)
+    return Instance(field_info, [set_type, get_type])
+
+
 def set_descriptor_types_for_field(ctx: FunctionContext) -> Instance:
     default_return_type = cast(Instance, ctx.default_return_type)
 
     is_nullable = False
-    null_expr = helpers.get_call_argument_by_name(ctx, 'null')
+    null_expr = chk_helpers.get_call_argument_by_name(ctx, 'null')
     if null_expr is not None:
         is_nullable = helpers.parse_bool(null_expr) or False
 
@@ -122,10 +127,10 @@ def set_descriptor_types_for_field(ctx: FunctionContext) -> Instance:
     return helpers.reparametrize_instance(default_return_type, [set_type, get_type])
 
 
-def determine_type_of_array_field(ctx: FunctionContext, django_context: DjangoContext) -> MypyType:
+def determine_type_of_array_field(ctx: FunctionContext) -> MypyType:
     default_return_type = set_descriptor_types_for_field(ctx)
 
-    base_field_arg_type = helpers.get_call_argument_type_by_name(ctx, 'base_field')
+    base_field_arg_type = chk_helpers.get_call_argument_type_by_name(ctx, 'base_field')
     if not base_field_arg_type or not isinstance(base_field_arg_type, Instance):
         return default_return_type
 
@@ -141,9 +146,9 @@ def transform_into_proper_return_type(ctx: FunctionContext, django_context: Djan
     default_return_type = ctx.default_return_type
     assert isinstance(default_return_type, Instance)
 
-    outer_model_info = helpers.get_typechecker_api(ctx).scope.active_class()
+    outer_model_info = chk_helpers.get_typechecker_api(ctx).scope.active_class()
     if (outer_model_info is None
-            or not helpers.is_model_subclass_info(outer_model_info, django_context)):
+            or not helpers.is_subclass_of_model(outer_model_info, django_context)):
         return ctx.default_return_type
 
     assert isinstance(outer_model_info, TypeInfo)
@@ -152,6 +157,6 @@ def transform_into_proper_return_type(ctx: FunctionContext, django_context: Djan
         return fill_descriptor_types_for_related_field(ctx, django_context)
 
     if default_return_type.type.has_base(fullnames.ARRAY_FIELD_FULLNAME):
-        return determine_type_of_array_field(ctx, django_context)
+        return determine_type_of_array_field(ctx)
 
     return set_descriptor_types_for_field(ctx)

mypy_django_plugin/transformers/forms.py

@@ -5,11 +5,11 @@ from mypy.types import CallableType, Instance, NoneTyp
 from mypy.types import Type as MypyType
 from mypy.types import TypeType
 
-from mypy_django_plugin.lib import helpers
+from mypy_django_plugin.lib import chk_helpers, sem_helpers
 
 
 def make_meta_nested_class_inherit_from_any(ctx: ClassDefContext) -> None:
-    meta_node = helpers.get_nested_meta_node_for_current_class(ctx.cls.info)
+    meta_node = sem_helpers.get_nested_meta_node_for_current_class(ctx.cls.info)
     if meta_node is None:
         if not ctx.api.final_iteration:
             ctx.api.defer()
@@ -28,7 +28,7 @@ def extract_proper_type_for_get_form(ctx: MethodContext) -> MypyType:
     object_type = ctx.type
     assert isinstance(object_type, Instance)
 
-    form_class_type = helpers.get_call_argument_type_by_name(ctx, 'form_class')
+    form_class_type = chk_helpers.get_call_argument_type_by_name(ctx, 'form_class')
     if form_class_type is None or isinstance(form_class_type, NoneTyp):
         form_class_type = get_specified_form_class(object_type)
 

mypy_django_plugin/transformers/init_create.py

@@ -6,7 +6,7 @@ from mypy.types import Instance
 from mypy.types import Type as MypyType
 
 from mypy_django_plugin.django.context import DjangoContext
-from mypy_django_plugin.lib import helpers
+from mypy_django_plugin.lib import chk_helpers
 
 
 def get_actual_types(ctx: Union[MethodContext, FunctionContext],
@@ -32,7 +32,7 @@ def get_actual_types(ctx: Union[MethodContext, FunctionContext],
 
 def typecheck_model_method(ctx: Union[FunctionContext, MethodContext], django_context: DjangoContext,
                            model_cls: Type[Model], method: str) -> MypyType:
-    typechecker_api = helpers.get_typechecker_api(ctx)
+    typechecker_api = chk_helpers.get_typechecker_api(ctx)
     expected_types = django_context.get_expected_types(typechecker_api, model_cls, method=method)
     expected_keys = [key for key in expected_types.keys() if key != 'pk']
 
@@ -42,11 +42,11 @@ def typecheck_model_method(ctx: Union[FunctionContext, MethodContext], django_co
                                                                            model_cls.__name__),
                          ctx.context)
             continue
-        helpers.check_types_compatible(ctx,
+        error_message = 'Incompatible type for "{}" of "{}"'.format(actual_name, model_cls.__name__)
+        chk_helpers.check_types_compatible(ctx,
                                            expected_type=expected_types[actual_name],
                                            actual_type=actual_type,
-                                       error_message='Incompatible type for "{}" of "{}"'.format(actual_name,
+                                           error_message=error_message)
-                                                                                                 model_cls.__name__))
 
     return ctx.default_return_type
 

mypy_django_plugin/transformers/managers.py

@@ -1,77 +1,232 @@
+from typing import Any, Dict, Iterator, Optional, Tuple
+
+from mypy.checker import gen_unique_name
 from mypy.nodes import (
-    GDEF, FuncDef, MemberExpr, NameExpr, RefExpr, StrExpr, SymbolTableNode, TypeInfo,
+    GDEF, CallExpr, Context, Decorator, FuncDef, MemberExpr, NameExpr, OverloadedFuncDef, PlaceholderNode, RefExpr,
-)
+    StrExpr, SymbolTable, SymbolTableNode, TypeInfo,
-from mypy.plugin import ClassDefContext, DynamicClassDefContext
+    MypyFile)
-from mypy.types import AnyType, Instance, TypeOfAny
+from mypy.plugin import ClassDefContext, DynamicClassDefContext, MethodContext
+from mypy.semanal import SemanticAnalyzer, is_same_symbol, is_valid_replacement
+from mypy.types import AnyType, CallableType, Instance, TypeVarType, TypeVarDef
+from mypy.types import Type as MypyType
+from mypy.types import TypeOfAny
 
-from mypy_django_plugin.lib import fullnames, helpers
+from mypy_django_plugin.lib import chk_helpers, fullnames, helpers, sem_helpers
 
 
-def create_new_manager_class_from_from_queryset_method(ctx: DynamicClassDefContext) -> None:
+def iter_all_custom_queryset_methods(derived_queryset_info: TypeInfo) -> Iterator[Tuple[str, FuncDef]]:
-    semanal_api = helpers.get_semanal_api(ctx)
+    for base_queryset_info in derived_queryset_info.mro:
+        if base_queryset_info.fullname == fullnames.QUERYSET_CLASS_FULLNAME:
+            break
+        for name, sym in base_queryset_info.names.items():
+            if isinstance(sym.node, FuncDef):
+                yield name, sym.node
 
-    callee = ctx.call.callee
+
+def generate_from_queryset_name(base_manager_info: TypeInfo, queryset_info: TypeInfo) -> str:
+    return base_manager_info.name + 'From' + queryset_info.name
+
+
+#
+# def cb_resolve_callee_info_or_exception(cb: ) -> TypeInfo:
+#     callee = ctx.call.callee
+#     assert isinstance(callee, MemberExpr)
+#     assert isinstance(callee.expr, RefExpr)
+#
+#     callee_info = callee.expr.node
+#     if (callee_info is None
+#             or isinstance(callee_info, PlaceholderNode)):
+#         raise sem_helpers.IncompleteDefnError(f'Definition of base manager {callee.fullname!r} '
+#                                                   f'is incomplete.')
+#
+#     assert isinstance(callee_info, TypeInfo)
+#     return callee_info
+
+
+def resolve_callee_info_or_exception(callback: helpers.DynamicClassPluginCallback) -> TypeInfo:
+    callee = callback.call_expr.callee
     assert isinstance(callee, MemberExpr)
     assert isinstance(callee.expr, RefExpr)
 
-    base_manager_info = callee.expr.node
+    callee_info = callee.expr.node
-    if base_manager_info is None:
+    if (callee_info is None
-        if not semanal_api.final_iteration:
+            or isinstance(callee_info, PlaceholderNode)):
-            semanal_api.defer()
+        raise sem_helpers.IncompleteDefnError(f'Definition of base manager {callee.fullname!r} '
-        return
+                                              f'is incomplete.')
 
-    assert isinstance(base_manager_info, TypeInfo)
+    assert isinstance(callee_info, TypeInfo)
-    new_manager_info = semanal_api.basic_new_typeinfo(ctx.name,
+    return callee_info
-                                                      basetype_or_fallback=Instance(base_manager_info,
-                                                                                    [AnyType(TypeOfAny.unannotated)]))
-    new_manager_info.line = ctx.call.line
-    new_manager_info.defn.line = ctx.call.line
-    new_manager_info.metaclass_type = new_manager_info.calculate_metaclass_type()
 
-    current_module = semanal_api.cur_mod_node
-    current_module.names[ctx.name] = SymbolTableNode(GDEF, new_manager_info,
-                                                     plugin_generated=True)
-    passed_queryset = ctx.call.args[0]
-    assert isinstance(passed_queryset, NameExpr)
 
-    derived_queryset_fullname = passed_queryset.fullname
+def resolve_passed_queryset_info_or_exception(callback: helpers.DynamicClassPluginCallback) -> TypeInfo:
-    assert derived_queryset_fullname is not None
+    passed_queryset_name_expr = callback.call_expr.args[0]
+    assert isinstance(passed_queryset_name_expr, NameExpr)
 
-    sym = semanal_api.lookup_fully_qualified_or_none(derived_queryset_fullname)
+    # lookup in the same module
-    assert sym is not None
+    sym = callback.semanal_api.lookup_qualified(passed_queryset_name_expr.name, ctx=callback.call_expr)
-    if sym.node is None:
+    if (sym is None
-        if not semanal_api.final_iteration:
+            or sym.node is None
-            semanal_api.defer()
+            or isinstance(sym.node, PlaceholderNode)):
-        else:
+        bound_name = passed_queryset_name_expr.fullname or passed_queryset_name_expr.name
-            # inherit from Any to prevent false-positives, if queryset class cannot be resolved
+        raise sem_helpers.BoundNameNotFound(bound_name)
-            new_manager_info.fallback_to_any = True
-        return
 
-    derived_queryset_info = sym.node
+    assert isinstance(sym.node, TypeInfo)
-    assert isinstance(derived_queryset_info, TypeInfo)
+    return sym.node
 
-    if len(ctx.call.args) > 1:
+
-        expr = ctx.call.args[1]
+def resolve_django_manager_info_or_exception(callback: helpers.DynamicClassPluginCallback) -> TypeInfo:
+    info = callback.lookup_typeinfo_or_defer(fullnames.MANAGER_CLASS_FULLNAME)
+    if info is None:
+        raise sem_helpers.BoundNameNotFound(fullnames.MANAGER_CLASS_FULLNAME)
+
+    return info
+
+
+def new_manager_typeinfo(ctx: DynamicClassDefContext, callee_manager_info: TypeInfo) -> TypeInfo:
+    callee_manager_type = Instance(callee_manager_info, [AnyType(TypeOfAny.unannotated)])
+    api = sem_helpers.get_semanal_api(ctx)
+
+    new_manager_class_name = ctx.name
+    new_manager_info = helpers.new_typeinfo(new_manager_class_name,
+                                            bases=[callee_manager_type], module_name=api.cur_mod_id)
+    new_manager_info.set_line(ctx.call)
+    return new_manager_info
+
+
+def get_generated_manager_fullname(call: CallExpr, base_manager_info: TypeInfo, queryset_info: TypeInfo) -> str:
+    if len(call.args) > 1:
+        # only for from_queryset()
+        expr = call.args[1]
         assert isinstance(expr, StrExpr)
         custom_manager_generated_name = expr.value
     else:
-        custom_manager_generated_name = base_manager_info.name + 'From' + derived_queryset_info.name
+        custom_manager_generated_name = base_manager_info.name + 'From' + queryset_info.name
 
-    custom_manager_generated_fullname = '.'.join(['django.db.models.manager', custom_manager_generated_name])
+    custom_manager_generated_fullname = 'django.db.models.manager' + '.' + custom_manager_generated_name
-    if 'from_queryset_managers' not in base_manager_info.metadata:
+    return custom_manager_generated_fullname
-        base_manager_info.metadata['from_queryset_managers'] = {}
+
-    base_manager_info.metadata['from_queryset_managers'][custom_manager_generated_fullname] = new_manager_info.fullname
+
+def get_generated_managers_metadata(django_manager_info: TypeInfo) -> Dict[str, Any]:
+    return django_manager_info.metadata.setdefault('from_queryset_managers', {})
+
+
+def record_new_manager_info_fullname_into_metadata(ctx: DynamicClassDefContext,
+                                                   new_manager_fullname: str,
+                                                   callee_manager_info: TypeInfo,
+                                                   queryset_info: TypeInfo,
+                                                   django_manager_info: TypeInfo) -> None:
+    custom_manager_generated_fullname = get_generated_manager_fullname(ctx.call,
+                                                                       base_manager_info=callee_manager_info,
+                                                                       queryset_info=queryset_info)
+    metadata = get_generated_managers_metadata(django_manager_info)
+    metadata[custom_manager_generated_fullname] = new_manager_fullname
+
+
+def create_new_manager_class_from_from_queryset_method(ctx: DynamicClassDefContext) -> None:
+    semanal_api = sem_helpers.get_semanal_api(ctx)
+    try:
+        callee_manager_info = resolve_callee_info_or_exception(ctx)
+        queryset_info = resolve_passed_queryset_info_or_exception(ctx)
+        django_manager_info = resolve_django_manager_info_or_exception(ctx)
+    except sem_helpers.IncompleteDefnError:
+        if not semanal_api.final_iteration:
+            semanal_api.defer()
+            return
+        else:
+            raise
+
+    new_manager_info = new_manager_typeinfo(ctx, callee_manager_info)
+    record_new_manager_info_fullname_into_metadata(ctx,
+                                                   new_manager_info.fullname,
+                                                   callee_manager_info,
+                                                   queryset_info,
+                                                   django_manager_info)
 
     class_def_context = ClassDefContext(cls=new_manager_info.defn,
                                         reason=ctx.call, api=semanal_api)
-    self_type = Instance(new_manager_info, [])
+    self_type = Instance(new_manager_info, [AnyType(TypeOfAny.explicit)])
-    # we need to copy all methods in MRO before django.db.models.query.QuerySet
+
-    for class_mro_info in derived_queryset_info.mro:
+    try:
-        if class_mro_info.fullname == fullnames.QUERYSET_CLASS_FULLNAME:
+        for name, method_node in iter_all_custom_queryset_methods(queryset_info):
-            break
+            sem_helpers.copy_method_or_incomplete_defn_exception(class_def_context,
-        for name, sym in class_mro_info.names.items():
-            if isinstance(sym.node, FuncDef):
-                helpers.copy_method_to_another_class(class_def_context,
                                                                  self_type,
                                                                  new_method_name=name,
-                                                     method_node=sym.node)
+                                                                 method_node=method_node)
+    except sem_helpers.IncompleteDefnError:
+        if not semanal_api.final_iteration:
+            semanal_api.defer()
+            return
+        else:
+            raise
+
+    new_manager_sym = SymbolTableNode(GDEF, new_manager_info, plugin_generated=True)
+
+    # context=None - forcibly replace old node
+    added = semanal_api.add_symbol_table_node(ctx.name, new_manager_sym, context=None)
+    if added:
+        # replace all references to the old manager Var everywhere
+        for _, module in semanal_api.modules.items():
+            if module.fullname != semanal_api.cur_mod_id:
+                for sym_name, sym in module.names.items():
+                    if sym.fullname == new_manager_info.fullname:
+                        module.names[sym_name] = new_manager_sym.copy()
+
+    # we need another iteration to process methods
+    if (not added
+            and not semanal_api.final_iteration):
+        semanal_api.defer()
+
+
+def add_symbol_table_node(api: SemanticAnalyzer,
+                          name: str,
+                          symbol: SymbolTableNode,
+                          context: Optional[Context] = None,
+                          symbol_table: Optional[SymbolTable] = None,
+                          can_defer: bool = True,
+                          escape_comprehensions: bool = False) -> bool:
+    """Add symbol table node to the currently active symbol table.
+
+    Return True if we actually added the symbol, or False if we refused
+    to do so (because something is not ready or it was a no-op).
+
+    Generate an error if there is an invalid redefinition.
+
+    If context is None, unconditionally add node, since we can't report
+    an error. Note that this is used by plugins to forcibly replace nodes!
+
+    TODO: Prevent plugins from replacing nodes, as it could cause problems?
+
+    Args:
+        name: short name of symbol
+        symbol: Node to add
+        can_defer: if True, defer current target if adding a placeholder
+        context: error context (see above about None value)
+    """
+    names = symbol_table or api.current_symbol_table(escape_comprehensions=escape_comprehensions)
+    existing = names.get(name)
+    if isinstance(symbol.node, PlaceholderNode) and can_defer:
+        api.defer(context)
+    if (existing is not None
+            and context is not None
+            and not is_valid_replacement(existing, symbol)):
+        # There is an existing node, so this may be a redefinition.
+        # If the new node points to the same node as the old one,
+        # or if both old and new nodes are placeholders, we don't
+        # need to do anything.
+        old = existing.node
+        new = symbol.node
+        if isinstance(new, PlaceholderNode):
+            # We don't know whether this is okay. Let's wait until the next iteration.
+            return False
+        if not is_same_symbol(old, new):
+            if isinstance(new, (FuncDef, Decorator, OverloadedFuncDef, TypeInfo)):
+                api.add_redefinition(names, name, symbol)
+            if not (isinstance(new, (FuncDef, Decorator))
+                    and api.set_original_def(old, new)):
+                api.name_already_defined(name, context, existing)
+    elif name not in api.missing_names and '*' not in api.missing_names:
+        names[name] = symbol
+        api.progress = True
+        return True
+    return False
+

mypy_django_plugin/transformers/meta.py

@@ -5,12 +5,12 @@ from mypy.types import Type as MypyType
 from mypy.types import TypeOfAny
 
 from mypy_django_plugin.django.context import DjangoContext
-from mypy_django_plugin.lib import helpers
+from mypy_django_plugin.lib import chk_helpers, helpers
 
 
 def _get_field_instance(ctx: MethodContext, field_fullname: str) -> MypyType:
-    field_info = helpers.lookup_fully_qualified_typeinfo(helpers.get_typechecker_api(ctx),
+    api = chk_helpers.get_typechecker_api(ctx)
-                                                         field_fullname)
+    field_info = helpers.lookup_fully_qualified_typeinfo(api, field_fullname)
     if field_info is None:
         return AnyType(TypeOfAny.unannotated)
     return Instance(field_info, [AnyType(TypeOfAny.explicit), AnyType(TypeOfAny.explicit)])
@@ -32,7 +32,7 @@ def return_proper_field_type_from_get_field(ctx: MethodContext, django_context:
     if model_cls is None:
         return ctx.default_return_type
 
-    field_name_expr = helpers.get_call_argument_by_name(ctx, 'field_name')
+    field_name_expr = chk_helpers.get_call_argument_by_name(ctx, 'field_name')
     if field_name_expr is None:
         return ctx.default_return_type
 

mypy_django_plugin/transformers/models.py

@@ -1,21 +1,23 @@
-from typing import Dict, List, Optional, Type, cast
+from typing import List, Optional, Type, cast
 
 from django.db.models.base import Model
 from django.db.models.fields import DateField, DateTimeField
-from django.db.models.fields.related import ForeignKey
+from django.db.models.fields.related import ForeignKey, OneToOneField
 from django.db.models.fields.reverse_related import (
     ManyToManyRel, ManyToOneRel, OneToOneRel,
 )
-from mypy.nodes import ARG_STAR2, Argument, Context, FuncDef, TypeInfo, Var
+from mypy.nodes import (
+    ARG_STAR2, GDEF, MDEF, Argument, Context, SymbolTableNode, TypeInfo, Var,
+)
 from mypy.plugin import ClassDefContext
 from mypy.plugins import common
-from mypy.semanal import SemanticAnalyzer
+from mypy.semanal import SemanticAnalyzer, dummy_context
 from mypy.types import AnyType, Instance
 from mypy.types import Type as MypyType
 from mypy.types import TypeOfAny
 
 from mypy_django_plugin.django.context import DjangoContext
-from mypy_django_plugin.lib import fullnames, helpers
+from mypy_django_plugin.lib import fullnames, helpers, sem_helpers
 from mypy_django_plugin.transformers import fields
 from mypy_django_plugin.transformers.fields import get_field_descriptor_types
 
@@ -32,37 +34,63 @@ class ModelClassInitializer:
     def lookup_typeinfo(self, fullname: str) -> Optional[TypeInfo]:
         return helpers.lookup_fully_qualified_typeinfo(self.api, fullname)
 
-    def lookup_typeinfo_or_incomplete_defn_error(self, fullname: str) -> TypeInfo:
+    def lookup_typeinfo_or_exception(self, fullname: str) -> TypeInfo:
         info = self.lookup_typeinfo(fullname)
         if info is None:
-            raise helpers.IncompleteDefnException(f'No {fullname!r} found')
+            raise sem_helpers.IncompleteDefnError(f'No {fullname!r} found')
         return info
 
-    def lookup_class_typeinfo_or_incomplete_defn_error(self, klass: type) -> TypeInfo:
+    def lookup_class_typeinfo_or_exception(self, klass: type) -> TypeInfo:
         fullname = helpers.get_class_fullname(klass)
-        field_info = self.lookup_typeinfo_or_incomplete_defn_error(fullname)
+        field_info = self.lookup_typeinfo_or_exception(fullname)
         return field_info
 
-    def create_new_var(self, name: str, typ: MypyType) -> Var:
+    def model_class_has_attribute_defined(self, name: str, traverse_mro: bool = True) -> bool:
-        # type=: type of the variable itself
+        if not traverse_mro:
-        var = Var(name=name, type=typ)
+            sym = self.model_classdef.info.names.get(name)
-        # var.info: type of the object variable is bound to
+        else:
+            sym = self.model_classdef.info.get(name)
+        return sym is not None
+
+    def resolve_manager_fullname(self, manager_fullname: str) -> str:
+        base_manager_info = self.lookup_typeinfo(fullnames.MANAGER_CLASS_FULLNAME)
+        if (base_manager_info is None
+                or 'from_queryset_managers' not in base_manager_info.metadata):
+            return manager_fullname
+
+        metadata = base_manager_info.metadata['from_queryset_managers']
+        return metadata.get(manager_fullname, manager_fullname)
+
+    def add_new_node_to_model_class(self, name: str, typ: MypyType,
+                                    force_replace_existing: bool = False) -> None:
+        if not force_replace_existing and name in self.model_classdef.info.names:
+            raise ValueError(f'Member {name!r} already defined at model {self.model_classdef.info.fullname!r}.')
+
+        var = Var(name, type=typ)
+        # TypeInfo of the object variable is bound to
         var.info = self.model_classdef.info
-        var._fullname = self.model_classdef.info.fullname + '.' + name
+        var._fullname = self.api.qualified_name(name)
         var.is_initialized_in_class = True
-        var.is_inferred = True
-        return var
 
-    def add_new_node_to_model_class(self, name: str, typ: MypyType) -> None:
+        sym = SymbolTableNode(MDEF, var, plugin_generated=True)
-        helpers.add_new_sym_for_info(self.model_classdef.info,
+        context: Optional[Context] = dummy_context()
-                                     name=name,
+        if force_replace_existing:
-                                     sym_type=typ)
+            context = None
+        self.api.add_symbol_table_node(name, sym, context=context)
 
-    def add_new_class_for_current_module(self, name: str, bases: List[Instance]) -> TypeInfo:
+    def add_new_class_for_current_module(self, name: str, bases: List[Instance],
-        current_module = self.api.modules[self.model_classdef.info.module_name]
+                                         force_replace_existing: bool = False) -> TypeInfo:
-        new_class_info = helpers.add_new_class_for_module(current_module,
+        current_module = self.api.cur_mod_node
-                                                          name=name, bases=bases)
+        if not force_replace_existing and name in current_module.names:
-        return new_class_info
+            raise ValueError(f'Class {name!r} already defined for module {current_module.fullname!r}')
+
+        new_typeinfo = helpers.new_typeinfo(name,
+                                            bases=bases,
+                                            module_name=current_module.fullname)
+        if name in current_module.names:
+            del current_module.names[name]
+        current_module.names[name] = SymbolTableNode(GDEF, new_typeinfo, plugin_generated=True)
+        return new_typeinfo
 
     def run(self) -> None:
         model_cls = self.django_context.get_model_class_by_fullname(self.model_classdef.fullname)
@@ -88,29 +116,59 @@ class InjectAnyAsBaseForNestedMeta(ModelClassInitializer):
     """
 
     def run(self) -> None:
-        meta_node = helpers.get_nested_meta_node_for_current_class(self.model_classdef.info)
+        meta_node = sem_helpers.get_nested_meta_node_for_current_class(self.model_classdef.info)
         if meta_node is None:
             return None
         meta_node.fallback_to_any = True
 
 
 class AddDefaultPrimaryKey(ModelClassInitializer):
+    """
+    Adds default primary key to models which does not define their own.
+    ```
+        class User(models.Model):
+            name = models.TextField()
+    ```
+    """
+
     def run_with_model_cls(self, model_cls: Type[Model]) -> None:
         auto_field = model_cls._meta.auto_field
-        if auto_field and not self.model_classdef.info.has_readable_member(auto_field.attname):
+        if auto_field is None:
-            # autogenerated field
+            return
-            auto_field_fullname = helpers.get_class_fullname(auto_field.__class__)
+
-            auto_field_info = self.lookup_typeinfo_or_incomplete_defn_error(auto_field_fullname)
+        primary_key_attrname = auto_field.attname
+        if self.model_class_has_attribute_defined(primary_key_attrname):
+            return
+
+        auto_field_class_fullname = helpers.get_class_fullname(auto_field.__class__)
+        auto_field_info = self.lookup_typeinfo_or_exception(auto_field_class_fullname)
 
         set_type, get_type = fields.get_field_descriptor_types(auto_field_info, is_nullable=False)
-            self.add_new_node_to_model_class(auto_field.attname, Instance(auto_field_info,
+        self.add_new_node_to_model_class(primary_key_attrname, Instance(auto_field_info,
                                                                         [set_type, get_type]))
 
 
 class AddRelatedModelsId(ModelClassInitializer):
+    """
+    Adds `FIELDNAME_id` attributes to models.
+    ```
+        class User(models.Model):
+            pass
+        class Blog(models.Model):
+            user = models.ForeignKey(User)
+    ```
+
+     `user_id` will be added to `Blog`.
+    """
+
     def run_with_model_cls(self, model_cls: Type[Model]) -> None:
         for field in model_cls._meta.get_fields():
-            if isinstance(field, ForeignKey):
+            if not isinstance(field, (OneToOneField, ForeignKey)):
+                continue
+            related_id_attr_name = field.attname
+            if self.model_class_has_attribute_defined(related_id_attr_name):
+                continue
+
             related_model_cls = self.django_context.get_field_related_model_cls(field)
             if related_model_cls is None:
                 error_context: Context = self.ctx.cls
@@ -120,7 +178,7 @@ class AddRelatedModelsId(ModelClassInitializer):
                 self.api.fail(f'Cannot find model {field.related_model!r} '
                               f'referenced in field {field.name!r} ',
                               ctx=error_context)
-                    self.add_new_node_to_model_class(field.attname,
+                self.add_new_node_to_model_class(related_id_attr_name,
                                                  AnyType(TypeOfAny.explicit))
                 continue
 
@@ -129,8 +187,8 @@ class AddRelatedModelsId(ModelClassInitializer):
 
             rel_primary_key_field = self.django_context.get_primary_key_field(related_model_cls)
             try:
-                    field_info = self.lookup_class_typeinfo_or_incomplete_defn_error(rel_primary_key_field.__class__)
+                field_info = self.lookup_class_typeinfo_or_exception(rel_primary_key_field.__class__)
-                except helpers.IncompleteDefnException as exc:
+            except sem_helpers.IncompleteDefnError as exc:
                 if not self.api.final_iteration:
                     raise exc
                 else:
@@ -138,109 +196,53 @@ class AddRelatedModelsId(ModelClassInitializer):
 
             is_nullable = self.django_context.get_field_nullability(field, None)
             set_type, get_type = get_field_descriptor_types(field_info, is_nullable)
-                self.add_new_node_to_model_class(field.attname,
+            self.add_new_node_to_model_class(related_id_attr_name,
                                              Instance(field_info, [set_type, get_type]))
 
 
 class AddManagers(ModelClassInitializer):
-    def has_any_parametrized_manager_as_base(self, info: TypeInfo) -> bool:
-        for base in helpers.iter_bases(info):
-            if self.is_any_parametrized_manager(base):
-                return True
-        return False
-
-    def is_any_parametrized_manager(self, typ: Instance) -> bool:
-        return typ.type.fullname in fullnames.MANAGER_CLASSES and isinstance(typ.args[0], AnyType)
-
-    def get_generated_manager_mappings(self, base_manager_fullname: str) -> Dict[str, str]:
-        base_manager_info = self.lookup_typeinfo(base_manager_fullname)
-        if (base_manager_info is None
-                or 'from_queryset_managers' not in base_manager_info.metadata):
-            return {}
-        return base_manager_info.metadata['from_queryset_managers']
-
-    def create_new_model_parametrized_manager(self, name: str, base_manager_info: TypeInfo) -> Instance:
-        bases = []
-        for original_base in base_manager_info.bases:
-            if self.is_any_parametrized_manager(original_base):
-                if original_base.type is None:
-                    raise helpers.IncompleteDefnException()
-
-                original_base = helpers.reparametrize_instance(original_base,
-                                                               [Instance(self.model_classdef.info, [])])
-            bases.append(original_base)
-
-        new_manager_info = self.add_new_class_for_current_module(name, bases)
-        # copy fields to a new manager
-        new_cls_def_context = ClassDefContext(cls=new_manager_info.defn,
-                                              reason=self.ctx.reason,
-                                              api=self.api)
-        custom_manager_type = Instance(new_manager_info, [Instance(self.model_classdef.info, [])])
-
-        for name, sym in base_manager_info.names.items():
-            # replace self type with new class, if copying method
-            if isinstance(sym.node, FuncDef):
-                helpers.copy_method_to_another_class(new_cls_def_context,
-                                                     self_type=custom_manager_type,
-                                                     new_method_name=name,
-                                                     method_node=sym.node)
-                continue
-
-            new_sym = sym.copy()
-            if isinstance(new_sym.node, Var):
-                new_var = Var(name, type=sym.type)
-                new_var.info = new_manager_info
-                new_var._fullname = new_manager_info.fullname + '.' + name
-                new_sym.node = new_var
-            new_manager_info.names[name] = new_sym
-
-        return custom_manager_type
-
     def run_with_model_cls(self, model_cls: Type[Model]) -> None:
         for manager_name, manager in model_cls._meta.managers_map.items():
-            manager_class_name = manager.__class__.__name__
+            if self.model_class_has_attribute_defined(manager_name, traverse_mro=False):
-            manager_fullname = helpers.get_class_fullname(manager.__class__)
+                sym = self.model_classdef.info.names.get(manager_name)
-            try:
+                assert sym is not None
-                manager_info = self.lookup_typeinfo_or_incomplete_defn_error(manager_fullname)
+
-            except helpers.IncompleteDefnException as exc:
+                if (sym.type is not None
-                if not self.api.final_iteration:
+                        and isinstance(sym.type, Instance)
-                    raise exc
+                        and sym.type.type.has_base(fullnames.BASE_MANAGER_CLASS_FULLNAME)):
-                else:
+                    # already defined and parametrized properly
-                    base_manager_fullname = helpers.get_class_fullname(manager.__class__.__bases__[0])
-                    generated_managers = self.get_generated_manager_mappings(base_manager_fullname)
-                    if manager_fullname not in generated_managers:
-                        # not a generated manager, continue with the loop
                     continue
-                    real_manager_fullname = generated_managers[manager_fullname]
+
-                    manager_info = self.lookup_typeinfo(real_manager_fullname)  # type: ignore
+            if getattr(manager, '_built_with_as_manager', False):
-                    if manager_info is None:
+                # as_manager is not supported yet
+                if not self.model_class_has_attribute_defined(manager_name, traverse_mro=True):
+                    self.add_new_node_to_model_class(manager_name, AnyType(TypeOfAny.explicit))
                 continue
-                    manager_class_name = real_manager_fullname.rsplit('.', maxsplit=1)[1]
+
+            manager_fullname = self.resolve_manager_fullname(helpers.get_class_fullname(manager.__class__))
+            manager_info = self.lookup_typeinfo_or_exception(manager_fullname)
 
             if manager_name not in self.model_classdef.info.names:
+                # manager not yet defined, just add models.Manager[ModelName]
                 manager_type = Instance(manager_info, [Instance(self.model_classdef.info, [])])
                 self.add_new_node_to_model_class(manager_name, manager_type)
-            else:
-                # creates new MODELNAME_MANAGERCLASSNAME class that represents manager parametrized with current model
-                if not self.has_any_parametrized_manager_as_base(manager_info):
-                    continue
-
-                custom_model_manager_name = manager.model.__name__ + '_' + manager_class_name
-                try:
-                    custom_manager_type = self.create_new_model_parametrized_manager(custom_model_manager_name,
-                                                                                     base_manager_info=manager_info)
-                except helpers.IncompleteDefnException:
-                    continue
-
-                self.add_new_node_to_model_class(manager_name, custom_manager_type)
 
 
 class AddDefaultManagerAttribute(ModelClassInitializer):
     def run_with_model_cls(self, model_cls: Type[Model]) -> None:
-        # add _default_manager
+        if self.model_class_has_attribute_defined('_default_manager', traverse_mro=False):
-        if '_default_manager' not in self.model_classdef.info.names:
+            return
+        if model_cls._meta.default_manager is None:
+            return
+        if getattr(model_cls._meta.default_manager, '_built_with_as_manager', False):
+            self.add_new_node_to_model_class('_default_manager',
+                                             AnyType(TypeOfAny.explicit))
+            return
+
         default_manager_fullname = helpers.get_class_fullname(model_cls._meta.default_manager.__class__)
-            default_manager_info = self.lookup_typeinfo_or_incomplete_defn_error(default_manager_fullname)
+        resolved_default_manager_fullname = self.resolve_manager_fullname(default_manager_fullname)
+
+        default_manager_info = self.lookup_typeinfo_or_exception(resolved_default_manager_fullname)
         default_manager = Instance(default_manager_info, [Instance(self.model_classdef.info, [])])
         self.add_new_node_to_model_class('_default_manager', default_manager)
 
@@ -249,33 +251,38 @@ class AddRelatedManagers(ModelClassInitializer):
     def run_with_model_cls(self, model_cls: Type[Model]) -> None:
         # add related managers
         for relation in self.django_context.get_model_relations(model_cls):
-            attname = relation.get_accessor_name()
+            related_manager_attr_name = relation.get_accessor_name()
-            if attname is None:
+            if related_manager_attr_name is None:
                 # no reverse accessor
                 continue
 
+            if self.model_class_has_attribute_defined(related_manager_attr_name, traverse_mro=False):
+                continue
+
             related_model_cls = self.django_context.get_field_related_model_cls(relation)
             if related_model_cls is None:
+                # could not find a referenced model (maybe invalid to= value)
                 continue
 
             try:
-                related_model_info = self.lookup_class_typeinfo_or_incomplete_defn_error(related_model_cls)
+                related_model_info = self.lookup_class_typeinfo_or_exception(related_model_cls)
-            except helpers.IncompleteDefnException as exc:
+            except sem_helpers.IncompleteDefnError as exc:
                 if not self.api.final_iteration:
                     raise exc
                 else:
                     continue
 
             if isinstance(relation, OneToOneRel):
-                self.add_new_node_to_model_class(attname, Instance(related_model_info, []))
+                self.add_new_node_to_model_class(related_manager_attr_name, Instance(related_model_info, []))
                 continue
 
             if isinstance(relation, (ManyToOneRel, ManyToManyRel)):
                 try:
-                    related_manager_info = self.lookup_typeinfo_or_incomplete_defn_error(fullnames.RELATED_MANAGER_CLASS)  # noqa: E501
+                    related_manager_info = self.lookup_typeinfo_or_exception(
+                        fullnames.RELATED_MANAGER_CLASS)  # noqa: E501
                     if 'objects' not in related_model_info.names:
-                        raise helpers.IncompleteDefnException()
+                        raise sem_helpers.IncompleteDefnError()
-                except helpers.IncompleteDefnException as exc:
+                except sem_helpers.IncompleteDefnError as exc:
                     if not self.api.final_iteration:
                         raise exc
                     else:
@@ -288,14 +295,15 @@ class AddRelatedManagers(ModelClassInitializer):
                 if (default_manager_type is None
                         or not isinstance(default_manager_type, Instance)
                         or default_manager_type.type.fullname == fullnames.MANAGER_CLASS_FULLNAME):
-                    self.add_new_node_to_model_class(attname, parametrized_related_manager_type)
+                    self.add_new_node_to_model_class(related_manager_attr_name, parametrized_related_manager_type)
                     continue
 
                 name = related_model_cls.__name__ + '_' + 'RelatedManager'
                 bases = [parametrized_related_manager_type, default_manager_type]
-                new_related_manager_info = self.add_new_class_for_current_module(name, bases)
+                new_related_manager_info = self.add_new_class_for_current_module(name, bases,
-
+                                                                                 force_replace_existing=True)
-                self.add_new_node_to_model_class(attname, Instance(new_related_manager_info, []))
+                self.add_new_node_to_model_class(related_manager_attr_name,
+                                                 Instance(new_related_manager_info, []))
 
 
 class AddExtraFieldMethods(ModelClassInitializer):
@@ -303,7 +311,7 @@ class AddExtraFieldMethods(ModelClassInitializer):
         # get_FOO_display for choices
         for field in self.django_context.get_model_fields(model_cls):
             if field.choices:
-                info = self.lookup_typeinfo_or_incomplete_defn_error('builtins.str')
+                info = self.lookup_typeinfo_or_exception('builtins.str')
                 return_type = Instance(info, [])
                 common.add_method(self.ctx,
                                   name='get_{}_display'.format(field.attname),
@@ -333,7 +341,7 @@ class AddExtraFieldMethods(ModelClassInitializer):
 class AddMetaOptionsAttribute(ModelClassInitializer):
     def run_with_model_cls(self, model_cls: Type[Model]) -> None:
         if '_meta' not in self.model_classdef.info.names:
-            options_info = self.lookup_typeinfo_or_incomplete_defn_error(fullnames.OPTIONS_CLASS_FULLNAME)
+            options_info = self.lookup_typeinfo_or_exception(fullnames.OPTIONS_CLASS_FULLNAME)
             self.add_new_node_to_model_class('_meta',
                                              Instance(options_info, [
                                                  Instance(self.model_classdef.info, [])
@@ -355,6 +363,8 @@ def process_model_class(ctx: ClassDefContext,
     for initializer_cls in initializers:
         try:
             initializer_cls(ctx, django_context).run()
-        except helpers.IncompleteDefnException:
+        except sem_helpers.IncompleteDefnError as exc:
             if not ctx.api.final_iteration:
                 ctx.api.defer()
+                continue
+            raise exc

mypy_django_plugin/transformers/orm_lookups.py

@@ -4,7 +4,7 @@ from mypy.types import Type as MypyType
 from mypy.types import TypeOfAny
 
 from mypy_django_plugin.django.context import DjangoContext
-from mypy_django_plugin.lib import fullnames, helpers
+from mypy_django_plugin.lib import chk_helpers, fullnames, helpers
 
 
 def typecheck_queryset_filter(ctx: MethodContext, django_context: DjangoContext) -> MypyType:
@@ -35,7 +35,7 @@ def typecheck_queryset_filter(ctx: MethodContext, django_context: DjangoContext)
                                                                   fullnames.QUERYSET_CLASS_FULLNAME))):
             return ctx.default_return_type
 
-        helpers.check_types_compatible(ctx,
+        chk_helpers.check_types_compatible(ctx,
                                            expected_type=lookup_type,
                                            actual_type=provided_type,
                                            error_message=f'Incompatible type for lookup {lookup_kwarg!r}:')

mypy_django_plugin/transformers/querysets.py

@@ -14,7 +14,7 @@ from mypy.types import TypeOfAny
 from mypy_django_plugin.django.context import (
     DjangoContext, LookupsAreUnsupported,
 )
-from mypy_django_plugin.lib import fullnames, helpers
+from mypy_django_plugin.lib import chk_helpers, fullnames, helpers
 
 
 def _extract_model_type_from_queryset(queryset_type: Instance) -> Optional[Instance]:
@@ -30,7 +30,7 @@ def determine_proper_manager_type(ctx: FunctionContext) -> MypyType:
     default_return_type = ctx.default_return_type
     assert isinstance(default_return_type, Instance)
 
-    outer_model_info = helpers.get_typechecker_api(ctx).scope.active_class()
+    outer_model_info = chk_helpers.get_typechecker_api(ctx).scope.active_class()
     if (outer_model_info is None
             or not outer_model_info.has_base(fullnames.MODEL_CLASS_FULLNAME)):
         return default_return_type
@@ -55,7 +55,7 @@ def get_field_type_from_lookup(ctx: MethodContext, django_context: DjangoContext
             return AnyType(TypeOfAny.from_error)
         lookup_field = django_context.get_primary_key_field(related_model_cls)
 
-    field_get_type = django_context.get_field_get_type(helpers.get_typechecker_api(ctx),
+    field_get_type = django_context.get_field_get_type(chk_helpers.get_typechecker_api(ctx),
                                                        lookup_field, method=method)
     return field_get_type
 
@@ -66,7 +66,7 @@ def get_values_list_row_type(ctx: MethodContext, django_context: DjangoContext,
     if field_lookups is None:
         return AnyType(TypeOfAny.from_error)
 
-    typechecker_api = helpers.get_typechecker_api(ctx)
+    typechecker_api = chk_helpers.get_typechecker_api(ctx)
     if len(field_lookups) == 0:
         if flat:
             primary_key_field = django_context.get_primary_key_field(model_cls)
@@ -80,7 +80,7 @@ def get_values_list_row_type(ctx: MethodContext, django_context: DjangoContext,
                 column_type = django_context.get_field_get_type(typechecker_api, field,
                                                                 method='values_list')
                 column_types[field.attname] = column_type
-            return helpers.make_oneoff_named_tuple(typechecker_api, 'Row', column_types)
+            return chk_helpers.make_oneoff_named_tuple(typechecker_api, 'Row', column_types)
         else:
             # flat=False, named=False, all fields
             field_lookups = []
@@ -103,9 +103,9 @@ def get_values_list_row_type(ctx: MethodContext, django_context: DjangoContext,
         assert len(column_types) == 1
         row_type = next(iter(column_types.values()))
     elif named:
-        row_type = helpers.make_oneoff_named_tuple(typechecker_api, 'Row', column_types)
+        row_type = chk_helpers.make_oneoff_named_tuple(typechecker_api, 'Row', column_types)
     else:
-        row_type = helpers.make_tuple(typechecker_api, list(column_types.values()))
+        row_type = chk_helpers.make_tuple(typechecker_api, list(column_types.values()))
 
     return row_type
 
@@ -123,13 +123,13 @@ def extract_proper_type_queryset_values_list(ctx: MethodContext, django_context:
     if model_cls is None:
         return ctx.default_return_type
 
-    flat_expr = helpers.get_call_argument_by_name(ctx, 'flat')
+    flat_expr = chk_helpers.get_call_argument_by_name(ctx, 'flat')
     if flat_expr is not None and isinstance(flat_expr, NameExpr):
         flat = helpers.parse_bool(flat_expr)
     else:
         flat = False
 
-    named_expr = helpers.get_call_argument_by_name(ctx, 'named')
+    named_expr = chk_helpers.get_call_argument_by_name(ctx, 'named')
     if named_expr is not None and isinstance(named_expr, NameExpr):
         named = helpers.parse_bool(named_expr)
     else:
@@ -188,5 +188,5 @@ def extract_proper_type_queryset_values(ctx: MethodContext, django_context: Djan
 
         column_types[field_lookup] = field_lookup_type
 
-    row_type = helpers.make_typeddict(ctx.api, column_types, set(column_types.keys()))
+    row_type = chk_helpers.make_oneoff_typeddict(ctx.api, column_types, set(column_types.keys()))
     return helpers.reparametrize_instance(ctx.default_return_type, [model_type, row_type])

mypy_django_plugin/transformers/request.py

@@ -3,13 +3,13 @@ from mypy.types import Instance
 from mypy.types import Type as MypyType
 
 from mypy_django_plugin.django.context import DjangoContext
-from mypy_django_plugin.lib import helpers
+from mypy_django_plugin.lib import chk_helpers, helpers
 
 
 def set_auth_user_model_as_type_for_request_user(ctx: AttributeContext, django_context: DjangoContext) -> MypyType:
     auth_user_model = django_context.settings.AUTH_USER_MODEL
     model_cls = django_context.apps_registry.get_model(auth_user_model)
-    model_info = helpers.lookup_class_typeinfo(helpers.get_typechecker_api(ctx), model_cls)
+    model_info = helpers.lookup_class_typeinfo(chk_helpers.get_typechecker_api(ctx), model_cls)
     if model_info is None:
         return ctx.default_attr_type
 

mypy_django_plugin/transformers/settings.py

@@ -5,7 +5,7 @@ from mypy.types import Type as MypyType
 from mypy.types import TypeOfAny, TypeType
 
 from mypy_django_plugin.django.context import DjangoContext
-from mypy_django_plugin.lib import helpers
+from mypy_django_plugin.lib import chk_helpers, helpers
 
 
 def get_user_model_hook(ctx: FunctionContext, django_context: DjangoContext) -> MypyType:
@@ -13,7 +13,7 @@ def get_user_model_hook(ctx: FunctionContext, django_context: DjangoContext) ->
     model_cls = django_context.apps_registry.get_model(auth_user_model)
     model_cls_fullname = helpers.get_class_fullname(model_cls)
 
-    model_info = helpers.lookup_fully_qualified_typeinfo(helpers.get_typechecker_api(ctx),
+    model_info = helpers.lookup_fully_qualified_typeinfo(chk_helpers.get_typechecker_api(ctx),
                                                          model_cls_fullname)
     if model_info is None:
         return AnyType(TypeOfAny.unannotated)
@@ -28,7 +28,7 @@ def get_type_of_settings_attribute(ctx: AttributeContext, django_context: Django
         ctx.api.fail(f"'Settings' object has no attribute {setting_name!r}", ctx.context)
         return ctx.default_attr_type
 
-    typechecker_api = helpers.get_typechecker_api(ctx)
+    typechecker_api = chk_helpers.get_typechecker_api(ctx)
 
     # first look for the setting in the project settings file, then global settings
     settings_module = typechecker_api.modules.get(django_context.django_settings_module)

mypy_django_plugin/transformers2/dynamic_managers.py

@@ -0,0 +1,86 @@
+from typing import Optional
+
+from mypy.checker import gen_unique_name
+from mypy.nodes import NameExpr, TypeInfo, SymbolTableNode, StrExpr
+from mypy.types import Type as MypyType, TypeVarType, TypeVarDef, Instance
+
+from mypy_django_plugin.lib import helpers, fullnames
+from mypy_django_plugin.transformers.managers import iter_all_custom_queryset_methods
+
+
+class CreateNewManagerClassFrom_FromQuerySet(helpers.DynamicClassPluginCallback):
+    def set_manager_mapping(self, runtime_manager_fullname: str, generated_manager_fullname: str) -> None:
+        base_model_info = self.lookup_typeinfo_or_defer(fullnames.MODEL_CLASS_FULLNAME)
+        assert base_model_info is not None
+        managers_metadata = base_model_info.metadata.setdefault('managers', {})
+        managers_metadata[runtime_manager_fullname] = generated_manager_fullname
+
+    def create_typevar_in_current_module(self, name: str,
+                                         upper_bound: Optional[MypyType] = None) -> TypeVarDef:
+        tvar_name = gen_unique_name(name, self.semanal_api.globals)
+        tvar_def = TypeVarDef(tvar_name,
+                              fullname=self.semanal_api.cur_mod_id + '.' + tvar_name,
+                              id=-1,
+                              values=[],
+                              upper_bound=upper_bound)
+        return tvar_def
+
+    def create_new_dynamic_class(self) -> None:
+        # extract Manager class which will act as base
+        callee = self.get_callee()
+        fullname = callee.fullname or callee.expr.fullname
+        callee_manager_info = self.lookup_typeinfo_or_defer(fullname)
+        if callee_manager_info is None:
+            return None
+
+        # extract queryset from which we're going to copy methods
+        passed_queryset_name_expr = self.call_expr.args[0]
+        assert isinstance(passed_queryset_name_expr, NameExpr)
+        queryset_class_name = passed_queryset_name_expr.name
+        sym = self.lookup_same_module_or_defer(queryset_class_name)
+        if sym is None:
+            return None
+        assert isinstance(sym.node, TypeInfo)
+        passed_queryset_info = sym.node
+
+        # for TypeVar bound
+        base_model_info = self.lookup_typeinfo_or_defer(fullnames.MODEL_CLASS_FULLNAME)
+        if base_model_info is None:
+            return
+        model_tvar_defn = self.create_typevar_in_current_module('_M', upper_bound=Instance(base_model_info, []))
+        model_tvar_type = TypeVarType(model_tvar_defn)
+
+        # make Manager[_T]
+        parent_manager_type = Instance(callee_manager_info, [model_tvar_type])
+
+        # instantiate with a proper model, Manager[MyModel], filling all Manager type vars in process
+        queryset_type = Instance(passed_queryset_info, [Instance(base_model_info, [])])
+        new_manager_info = self.new_typeinfo(self.class_name,
+                                             bases=[queryset_type, parent_manager_type])
+        new_manager_info.defn.type_vars = [model_tvar_defn]
+        new_manager_info.type_vars = [model_tvar_defn.name]
+        new_manager_info.set_line(self.call_expr)
+
+        # copy methods from passed_queryset_info with self type replaced
+        # self_type = Instance(new_manager_info, [model_tvar_type])
+        # for name, method_node in iter_all_custom_queryset_methods(passed_queryset_info):
+        #     self.add_method_from_signature(method_node,
+        #                                    name,
+        #                                    self_type,
+        #                                    new_manager_info.defn)
+
+        new_manager_sym = SymbolTableNode(self.semanal_api.current_symbol_kind(),
+                                          new_manager_info,
+                                          plugin_generated=True)
+        self.semanal_api.add_symbol_table_node(self.class_name, new_manager_sym)
+
+        # add mapping between generated manager and current one
+        runtime_manager_class_name = None
+        if 'class_name' in self.call_expr.arg_names:
+            class_name_arg = self.call_expr.args[self.call_expr.arg_names.index('class_name')]
+            if isinstance(class_name_arg, StrExpr):
+                runtime_manager_class_name = class_name_arg.value
+
+        new_manager_name = runtime_manager_class_name or (callee_manager_info.name + 'From' + queryset_class_name)
+        self.set_manager_mapping(f'django.db.models.manager.{new_manager_name}',
+                                 new_manager_info.fullname)

mypy_django_plugin/transformers2/models.py

@@ -0,0 +1,303 @@
+from abc import abstractmethod
+from typing import Type, Optional
+
+from django.db.models.base import Model
+from django.db.models.fields.related import OneToOneField, ForeignKey
+from mypy.nodes import TypeInfo, Var, SymbolTableNode, MDEF, Argument, ARG_STAR2
+from mypy.plugin import ClassDefContext
+from mypy.plugins import common
+from mypy.semanal import dummy_context
+from mypy.types import Instance, TypeOfAny, AnyType
+from mypy.types import Type as MypyType
+
+from django.db import models
+from django.db.models.fields import DateField, DateTimeField
+from mypy_django_plugin.lib import helpers, fullnames, sem_helpers
+from mypy_django_plugin.transformers import fields
+from mypy_django_plugin.transformers.fields import get_field_type
+from mypy_django_plugin.transformers2 import new_helpers
+
+
+class TransformModelClassCallback(helpers.ClassDefPluginCallback):
+    def get_real_manager_fullname(self, manager_fullname: str) -> str:
+        model_info = self.lookup_typeinfo_or_defer(fullnames.MODEL_CLASS_FULLNAME)
+        real_manager_fullname = model_info.metadata.get('managers', {}).get(manager_fullname, manager_fullname)
+        return real_manager_fullname
+
+    def modify_class_defn(self) -> None:
+        model_cls = self.django_context.get_model_class_by_fullname(self.class_defn.fullname)
+        if model_cls is None:
+            return None
+        return self.modify_model_class_defn(model_cls)
+
+    def add_new_model_attribute(self, name: str, typ: MypyType, force_replace: bool = False) -> None:
+        model_info = self.class_defn.info
+        if name in model_info.names and not force_replace:
+            raise ValueError('Attribute already exists on the model')
+
+        var = Var(name, type=typ)
+        var.info = model_info
+        var._fullname = self.semanal_api.qualified_name(name)
+        var.is_initialized_in_class = True
+
+        sym = SymbolTableNode(MDEF, var, plugin_generated=True)
+        error_context = None if force_replace else dummy_context()
+        added = self.semanal_api.add_symbol_table_node(name, sym, context=error_context)
+        assert added
+
+    def lookup_typeinfo_for_class_or_defer(self, klass: type, *,
+                                           reason_for_defer: Optional[str] = None) -> Optional[TypeInfo]:
+        manager_cls_fullname = helpers.get_class_fullname(klass)
+        return self.lookup_typeinfo_or_defer(manager_cls_fullname,
+                                             reason_for_defer=reason_for_defer)
+
+    @abstractmethod
+    def modify_model_class_defn(self, runtime_model_cls: Type[Model]) -> None:
+        raise NotImplementedError
+
+
+class AddDefaultManagerCallback(TransformModelClassCallback):
+    def modify_model_class_defn(self, runtime_model_cls: Type[Model]) -> None:
+        if ('_default_manager' in self.class_defn.info.names
+                or runtime_model_cls._meta.default_manager is None):
+            return None
+
+        runtime_default_manager_class = runtime_model_cls._meta.default_manager.__class__
+        runtime_manager_cls_fullname = new_helpers.get_class_fullname(runtime_default_manager_class)
+        manager_cls_fullname = self.get_real_manager_fullname(runtime_manager_cls_fullname)
+
+        default_manager_info = self.lookup_typeinfo_or_defer(manager_cls_fullname)
+        if default_manager_info is None:
+            return
+
+        self.add_new_model_attribute('_default_manager',
+                                     Instance(default_manager_info, [Instance(self.class_defn.info, [])]))
+
+
+class AddManagersCallback(TransformModelClassCallback):
+    def modify_model_class_defn(self, runtime_model_cls: Type[models.Model]) -> None:
+        for manager_name, manager in runtime_model_cls._meta.managers_map.items():
+            if manager_name in self.class_defn.info.names:
+                # already defined on the current model class, in file or at a previous iteration
+                continue
+
+            manager_info = self.lookup_typeinfo_for_class_or_defer(manager.__class__)
+            if manager_info is None:
+                continue
+
+            manager_type = Instance(manager_info, [Instance(self.class_defn.info, [])])
+            self.add_new_model_attribute(manager_name, manager_type)
+
+
+class AddPrimaryKeyIfDoesNotExist(TransformModelClassCallback):
+    """
+    Adds default primary key to models which does not define their own.
+        class User(models.Model):
+            name = models.TextField()
+    """
+
+    def modify_model_class_defn(self, runtime_model_cls: Type[Model]) -> None:
+        auto_pk_field = runtime_model_cls._meta.auto_field
+        if auto_pk_field is None:
+            # defined explicitly
+            return None
+        auto_pk_field_name = auto_pk_field.attname
+        if auto_pk_field_name in self.class_defn.info.names:
+            # added on previous iteration
+            return None
+
+        auto_pk_field_info = self.lookup_typeinfo_for_class_or_defer(auto_pk_field.__class__)
+        if auto_pk_field_info is None:
+            return None
+
+        self.add_new_model_attribute(auto_pk_field_name,
+                                     fields.get_field_type(auto_pk_field_info, is_nullable=False))
+
+
+class AddRelatedManagersCallback(TransformModelClassCallback):
+    def modify_model_class_defn(self, runtime_model_cls: Type[Model]) -> None:
+        for reverse_manager_name, relation in self.django_context.get_model_relations(runtime_model_cls):
+            if (reverse_manager_name is None
+                    or reverse_manager_name in self.class_defn.info.names):
+                continue
+
+            self.add_new_model_attribute(reverse_manager_name, AnyType(TypeOfAny.implementation_artifact))
+            #
+            # related_model_cls = self.django_context.get_field_related_model_cls(relation)
+            # if related_model_cls is None:
+            #     # could not find a referenced model (maybe invalid to= value, or GenericForeignKey)
+            #     continue
+            #
+            # related_model_info = self.lookup_typeinfo_for_class_or_defer(related_model_cls)
+            # if related_model_info is None:
+            #     continue
+            #
+            # if isinstance(relation, OneToOneRel):
+            #     self.add_new_model_attribute(reverse_manager_name,
+            #                                  Instance(related_model_info, []))
+            # elif isinstance(relation, (ManyToOneRel, ManyToManyRel)):
+            #     related_manager_info = self.lookup_typeinfo_or_defer(fullnames.RELATED_MANAGER_CLASS)
+            #     if related_manager_info is None:
+            #         if not self.defer_till_next_iteration(self.class_defn,
+            #                                               reason=f'{fullnames.RELATED_MANAGER_CLASS!r} is not available for lookup'):
+            #             raise TypeInfoNotFound(fullnames.RELATED_MANAGER_CLASS)
+            #         continue
+            #
+            #     # get type of default_manager for model
+            #     default_manager_fullname = helpers.get_class_fullname(related_model_cls._meta.default_manager.__class__)
+            #     reason_for_defer = (f'Trying to lookup default_manager {default_manager_fullname!r} '
+            #                         f'of model {helpers.get_class_fullname(related_model_cls)!r}')
+            #     default_manager_info = self.lookup_typeinfo_or_defer(default_manager_fullname,
+            #                                                          reason_for_defer=reason_for_defer)
+            #     if default_manager_info is None:
+            #         continue
+            #
+            #     default_manager_type = Instance(default_manager_info, [Instance(related_model_info, [])])
+            #
+            #     # related_model_cls._meta.default_manager.__class__
+            #     # # we're making a subclass of 'objects', need to have it defined
+            #     # if 'objects' not in related_model_info.names:
+            #     #     if not self.defer_till_next_iteration(self.class_defn,
+            #     #                                           reason=f"'objects' manager is not yet defined on {related_model_info.fullname!r}"):
+            #     #         raise AttributeNotFound(self.class_defn.info, 'objects')
+            #     #     continue
+            #
+            #     related_manager_type = Instance(related_manager_info,
+            #                                     [Instance(related_model_info, [])])
+            #     #
+            #     # objects_sym = related_model_info.names['objects']
+            #     # default_manager_type = objects_sym.type
+            #     # if default_manager_type is None:
+            #     #     # dynamic base class, extract from django_context
+            #     #     default_manager_cls = related_model_cls._meta.default_manager.__class__
+            #     #     default_manager_info = self.lookup_typeinfo_for_class_or_defer(default_manager_cls)
+            #     #     if default_manager_info is None:
+            #     #         continue
+            #     #     default_manager_type = Instance(default_manager_info, [Instance(related_model_info, [])])
+            #
+            #     if (not isinstance(default_manager_type, Instance)
+            #             or default_manager_type.type.fullname == fullnames.MANAGER_CLASS_FULLNAME):
+            #         # if not defined or trivial -> just return RelatedManager[Model]
+            #         self.add_new_model_attribute(reverse_manager_name, related_manager_type)
+            #         continue
+            #
+            #     # make anonymous class
+            #     name = gen_unique_name(related_model_cls.__name__ + '_' + 'RelatedManager',
+            #                            self.semanal_api.current_symbol_table())
+            #     bases = [related_manager_type, default_manager_type]
+            #     new_manager_info = self.new_typeinfo(name, bases)
+            #     self.add_new_model_attribute(reverse_manager_name, Instance(new_manager_info, []))
+
+
+class AddForeignPrimaryKeys(TransformModelClassCallback):
+    def modify_model_class_defn(self, runtime_model_cls: Type[Model]) -> None:
+        for field in runtime_model_cls._meta.get_fields():
+            if not isinstance(field, (OneToOneField, ForeignKey)):
+                continue
+            rel_pk_field_name = field.attname
+            if rel_pk_field_name in self.class_defn.info.names:
+                continue
+
+            related_model_cls = self.django_context.get_field_related_model_cls(field)
+            if related_model_cls is None:
+                field_sym = self.class_defn.info.get(field.name)
+                if field_sym is not None and field_sym.node is not None:
+                    error_context = field_sym.node
+                else:
+                    error_context = self.class_defn
+                self.semanal_api.fail(f'Cannot find model {field.related_model!r} '
+                                      f'referenced in field {field.name!r} ',
+                                      ctx=error_context)
+                self.add_new_model_attribute(rel_pk_field_name, AnyType(TypeOfAny.from_error))
+                continue
+            if related_model_cls._meta.abstract:
+                continue
+
+            rel_pk_field = self.django_context.get_primary_key_field(related_model_cls)
+            rel_pk_field_info = self.lookup_typeinfo_for_class_or_defer(rel_pk_field.__class__)
+            if rel_pk_field_info is None:
+                continue
+
+            field_type = get_field_type(rel_pk_field_info,
+                                        is_nullable=self.django_context.get_field_nullability(field))
+            self.add_new_model_attribute(rel_pk_field_name, field_type)
+
+
+class InjectAnyAsBaseForNestedMeta(TransformModelClassCallback):
+    """
+    Replaces
+        class MyModel(models.Model):
+            class Meta:
+                pass
+    with
+        class MyModel(models.Model):
+            class Meta(Any):
+                pass
+    to get around incompatible Meta inner classes for different models.
+    """
+
+    def modify_class_defn(self) -> None:
+        meta_node = sem_helpers.get_nested_meta_node_for_current_class(self.class_defn.info)
+        if meta_node is None:
+            return None
+        meta_node.fallback_to_any = True
+
+
+class AddMetaOptionsAttribute(TransformModelClassCallback):
+    def modify_model_class_defn(self, runtime_model_cls: Type[Model]) -> None:
+        if '_meta' not in self.class_defn.info.names:
+            options_info = self.lookup_typeinfo_or_defer(fullnames.OPTIONS_CLASS_FULLNAME)
+            if options_info is not None:
+                self.add_new_model_attribute('_meta',
+                                             Instance(options_info, [
+                                                 Instance(self.class_defn.info, [])
+                                             ]))
+
+
+class AddExtraFieldMethods(TransformModelClassCallback):
+    def modify_model_class_defn(self, runtime_model_cls: Type[Model]) -> None:
+        # get_FOO_display for choices
+        for field in self.django_context.get_model_fields(runtime_model_cls):
+            if field.choices:
+                info = self.lookup_typeinfo_or_defer('builtins.str')
+                return_type = Instance(info, [])
+                common.add_method(self.ctx,
+                                  name='get_{}_display'.format(field.attname),
+                                  args=[],
+                                  return_type=return_type)
+
+        # get_next_by, get_previous_by for Date, DateTime
+        for field in self.django_context.get_model_fields(runtime_model_cls):
+            if isinstance(field, (DateField, DateTimeField)) and not field.null:
+                return_type = Instance(self.class_defn.info, [])
+                common.add_method(self.ctx,
+                                  name='get_next_by_{}'.format(field.attname),
+                                  args=[Argument(Var('kwargs', AnyType(TypeOfAny.explicit)),
+                                                 AnyType(TypeOfAny.explicit),
+                                                 initializer=None,
+                                                 kind=ARG_STAR2)],
+                                  return_type=return_type)
+                common.add_method(self.ctx,
+                                  name='get_previous_by_{}'.format(field.attname),
+                                  args=[Argument(Var('kwargs', AnyType(TypeOfAny.explicit)),
+                                                 AnyType(TypeOfAny.explicit),
+                                                 initializer=None,
+                                                 kind=ARG_STAR2)],
+                                  return_type=return_type)
+
+
+class ModelCallback(helpers.ClassDefPluginCallback):
+    def __call__(self, ctx: ClassDefContext) -> None:
+        callback_classes = [
+            AddManagersCallback,
+            AddPrimaryKeyIfDoesNotExist,
+            AddForeignPrimaryKeys,
+            AddDefaultManagerCallback,
+            AddRelatedManagersCallback,
+            InjectAnyAsBaseForNestedMeta,
+            AddMetaOptionsAttribute,
+            AddExtraFieldMethods,
+        ]
+        for callback_cls in callback_classes:
+            callback = callback_cls(self.plugin)
+            callback.__call__(ctx)

mypy_django_plugin/transformers2/new_helpers.py

@@ -0,0 +1,30 @@
+from typing import Union
+
+from mypy.nodes import TypeInfo, MypyFile
+
+
+class IncompleteDefnError(Exception):
+    pass
+
+
+class TypeInfoNotFound(IncompleteDefnError):
+    def __init__(self, fullname: str) -> None:
+        super().__init__(f'It is final iteration and required type {fullname!r} is not ready yet.')
+
+
+class AttributeNotFound(IncompleteDefnError):
+    def __init__(self, node: Union[TypeInfo, MypyFile], attrname: str) -> None:
+        super().__init__(f'Attribute {attrname!r} is not defined for the {node.fullname!r}.')
+
+
+class NameNotFound(IncompleteDefnError):
+    def __init__(self, name: str) -> None:
+        super().__init__(f'Could not find {name!r} in the current activated namespaces')
+
+
+class SymbolAdditionNotPossible(Exception):
+    pass
+
+
+def get_class_fullname(klass: type) -> str:
+    return klass.__module__ + '.' + klass.__qualname__

mypy_django_plugin/transformers2/related_managers.py

@@ -0,0 +1,69 @@
+from mypy.checker import gen_unique_name
+from mypy.plugin import AttributeContext
+from mypy.types import Instance
+from mypy.types import Type as MypyType
+
+from django.db.models.fields.reverse_related import ForeignObjectRel, OneToOneRel, ManyToOneRel, ManyToManyRel
+
+from mypy_django_plugin.lib import helpers, fullnames
+from mypy_django_plugin.lib.helpers import GetAttributeCallback
+
+
+class GetRelatedManagerCallback(GetAttributeCallback):
+    obj_type: Instance
+
+    def get_related_manager_type(self, relation: ForeignObjectRel) -> MypyType:
+        related_model_cls = self.django_context.get_field_related_model_cls(relation)
+        if related_model_cls is None:
+            # could not find a referenced model (maybe invalid to= value, or GenericForeignKey)
+            # TODO: show error
+            return self.default_attr_type
+
+        related_model_info = self.lookup_typeinfo(helpers.get_class_fullname(related_model_cls))
+        if related_model_info is None:
+            # TODO: show error
+            return self.default_attr_type
+
+        if isinstance(relation, OneToOneRel):
+            return Instance(related_model_info, [])
+
+        elif isinstance(relation, (ManyToOneRel, ManyToManyRel)):
+            related_manager_info = self.lookup_typeinfo(fullnames.RELATED_MANAGER_CLASS)
+            if related_manager_info is None:
+                return self.default_attr_type
+
+            # get type of default_manager for model
+            default_manager_fullname = helpers.get_class_fullname(related_model_cls._meta.default_manager.__class__)
+            default_manager_info = self.lookup_typeinfo(default_manager_fullname)
+            if default_manager_info is None:
+                return self.default_attr_type
+
+            default_manager_type = Instance(default_manager_info, [Instance(related_model_info, [])])
+            related_manager_type = Instance(related_manager_info,
+                                            [Instance(related_model_info, [])])
+
+            if (not isinstance(default_manager_type, Instance)
+                    or default_manager_type.type.fullname == fullnames.MANAGER_CLASS_FULLNAME):
+                # if not defined or trivial -> just return RelatedManager[Model]
+                return related_manager_type
+
+            # make anonymous class
+            name = gen_unique_name(related_model_cls.__name__ + '_' + 'RelatedManager',
+                                   self.obj_type.type.names)
+            bases = [related_manager_type, default_manager_type]
+            new_manager_info = self.new_typeinfo(name, bases)
+            return Instance(new_manager_info, [])
+
+    def __call__(self, ctx: AttributeContext):
+        super().__call__(ctx)
+        assert isinstance(self.obj_type, Instance)
+
+        model_fullname = self.obj_type.type.fullname
+        model_cls = self.django_context.get_model_class_by_fullname(model_fullname)
+        if model_cls is None:
+            return self.default_attr_type
+        for reverse_manager_name, relation in self.django_context.get_model_relations(model_cls):
+            if reverse_manager_name == self.name:
+                return self.get_related_manager_type(relation)
+
+        return self.default_attr_type

scripts/enabled_test_modules.py

@@ -314,6 +314,9 @@ IGNORED_ERRORS = {
     'model_enums': [
         "'bool' is not a valid base class",
     ],
+    'multiple_database': [
+        'Unexpected attribute "extra_arg" for model "Book"',
+    ],
     'null_queries': [
         "Cannot resolve keyword 'foo' into field"
     ],
@@ -457,7 +460,7 @@ IGNORED_ERRORS = {
         'No overload variant of "join" matches argument types "str", "None"',
         'Argument 1 to "Archive" has incompatible type "None"; expected "str"',
         'Argument 1 to "to_path" has incompatible type "int"; expected "Union[Path, str]"',
-        'Cannot infer type argument 1 of "cached_property"',
+
     ],
     'view_tests': [
         "Module 'django.views.debug' has no attribute 'Path'",

test-data/typecheck/fields/test_related.yml

@@ -36,7 +36,7 @@
                     pass
                 class Book(models.Model):
                     publisher = models.ForeignKey(to=Publisher, on_delete=models.CASCADE)
-                    owner = models.ForeignKey(db_column='model_id', to='auth.User', on_delete=models.CASCADE)
+                    owner = models.ForeignKey(to='auth.User', on_delete=models.CASCADE)
 
 -   case: foreign_key_field_different_order_of_params
     main: |
@@ -653,7 +653,7 @@
 -   case: related_manager_is_a_subclass_of_default_manager
     main: |
         from myapp.models import User
-        reveal_type(User().orders)  # N: Revealed type is 'myapp.models.Order_RelatedManager'
+        reveal_type(User().orders)  # N: Revealed type is 'main.Order_RelatedManager'
         reveal_type(User().orders.get())  # N: Revealed type is 'myapp.models.Order*'
         reveal_type(User().orders.manager_method())  # N: Revealed type is 'builtins.int'
     installed_apps:
@@ -663,9 +663,12 @@
         -   path: myapp/models.py
             content: |
                 from django.db import models
+                from mypy_django_plugin.lib import generics
+                generics.make_classes_generic(models.Manager)
+
                 class User(models.Model):
                     pass
-                class OrderManager(models.Manager):
+                class OrderManager(models.Manager['Order']):
                     def manager_method(self) -> int:
                         pass
                 class Order(models.Model):

test-data/typecheck/managers/querysets/test_from_queryset.yml

@@ -1,31 +1,46 @@
 -   case: from_queryset_with_base_manager
     main: |
         from myapp.models import MyModel
-        reveal_type(MyModel().objects)  # N: Revealed type is 'myapp.models.MyModel_NewManager[myapp.models.MyModel]'
+
+        reveal_type(MyModel().objects)  # N: Revealed type is 'myapp.models.NewBaseManager[myapp.models.MyModel*]'
         reveal_type(MyModel().objects.get())  # N: Revealed type is 'myapp.models.MyModel*'
+        reveal_type(MyModel().objects.queryset_method)  # N: Revealed type is 'def () -> builtins.str'
         reveal_type(MyModel().objects.queryset_method())  # N: Revealed type is 'builtins.str'
+
+        reveal_type(MyModel().objects2)  # N: Revealed type is 'myapp.models.NewManager[myapp.models.MyModel*]'
+        reveal_type(MyModel().objects2.get())  # N: Revealed type is 'myapp.models.MyModel*'
+        reveal_type(MyModel().objects2.queryset_method)  # N: Revealed type is 'def () -> builtins.str'
+        reveal_type(MyModel().objects2.queryset_method())  # N: Revealed type is 'builtins.str'
     installed_apps:
         - myapp
     files:
         -   path: myapp/__init__.py
         -   path: myapp/models.py
             content: |
+                from typing import TypeVar
                 from django.db import models
-                from django.db.models.manager import BaseManager
+                from django.db.models.manager import BaseManager, Manager
+                from mypy_django_plugin.lib import generics
 
-                class ModelQuerySet(models.QuerySet):
+                generics.make_classes_generic(models.QuerySet)
+                _M = TypeVar('_M', bound=models.Model)
+
+                class ModelQuerySet(models.QuerySet[_M]):
                     def queryset_method(self) -> str:
                         return 'hello'
-                NewManager = BaseManager.from_queryset(ModelQuerySet)
-                class MyModel(models.Model):
-                    objects = NewManager()
 
--   case: from_queryset_with_manager
+                NewBaseManager = BaseManager.from_queryset(ModelQuerySet)
+                NewManager = Manager.from_queryset(ModelQuerySet)
+
+                generics.make_classes_generic(NewBaseManager, NewManager)
+
+                class MyModel(models.Model):
+                    objects = NewBaseManager['MyModel']()
+                    objects2 = NewManager['MyModel']()
+
+-   case: manager_without_generic_requires_annotation
     main: |
-        from myapp.models import MyModel
+        from myapp.models import ModelQuerySet
-        reveal_type(MyModel().objects)  # N: Revealed type is 'myapp.models.MyModel_NewManager[myapp.models.MyModel]'
-        reveal_type(MyModel().objects.get())  # N: Revealed type is 'myapp.models.MyModel*'
-        reveal_type(MyModel().objects.queryset_method())  # N: Revealed type is 'builtins.str'
     installed_apps:
         - myapp
     files:
@@ -33,20 +48,27 @@
         -   path: myapp/models.py
             content: |
                 from django.db import models
+                from django.db.models.manager import BaseManager, Manager
+                from mypy_django_plugin.lib import generics
 
                 class ModelQuerySet(models.QuerySet):
                     def queryset_method(self) -> str:
                         return 'hello'
 
-                NewManager = models.Manager.from_queryset(ModelQuerySet)
+                NewBaseManager = BaseManager.from_queryset(ModelQuerySet)
-                class MyModel(models.Model):
+                NewManager = Manager.from_queryset(ModelQuerySet)
-                    objects = NewManager()
 
--   case: from_queryset_returns_intersection_of_manager_and_queryset
+                generics.make_classes_generic(NewBaseManager, NewManager)
+
+                class MyModel(models.Model):
+                    objects = NewBaseManager()  # E: Need type annotation for 'objects'
+
+
+-   case: from_queryset_with_custom_manager_as_base
     main: |
         from myapp.models import MyModel, NewManager
-        reveal_type(NewManager())  # N: Revealed type is 'myapp.models.NewManager'
+        reveal_type(NewManager())  # N: Revealed type is 'myapp.models.NewManager[<nothing>]'
-        reveal_type(MyModel.objects)  # N: Revealed type is 'myapp.models.MyModel_NewManager[myapp.models.MyModel]'
+        reveal_type(MyModel.objects)  # N: Revealed type is 'myapp.models.NewManager[myapp.models.MyModel]'
         reveal_type(MyModel.objects.get())  # N: Revealed type is 'Any'
         reveal_type(MyModel.objects.manager_only_method())  # N: Revealed type is 'builtins.int'
         reveal_type(MyModel.objects.manager_and_queryset_method())  # N: Revealed type is 'builtins.str'
@@ -57,6 +79,8 @@
         -   path: myapp/models.py
             content: |
                 from django.db import models
+                from mypy_django_plugin.lib import generics
+
                 class ModelBaseManager(models.Manager):
                     def manager_only_method(self) -> int:
                         return 1
@@ -65,14 +89,17 @@
                         return 'hello'
 
                 NewManager = ModelBaseManager.from_queryset(ModelQuerySet)
+                generics.make_classes_generic(NewManager)
+
                 class MyModel(models.Model):
-                    objects = NewManager()
+                    objects = NewManager['MyModel']()
+
 
 -   case: from_queryset_with_class_name_provided
     main: |
         from myapp.models import MyModel, NewManager
-        reveal_type(NewManager())  # N: Revealed type is 'myapp.models.NewManager'
+        reveal_type(NewManager())  # N: Revealed type is 'myapp.models.NewManager[<nothing>]'
-        reveal_type(MyModel.objects)  # N: Revealed type is 'myapp.models.MyModel_NewManager[myapp.models.MyModel]'
+        reveal_type(MyModel.objects)  # N: Revealed type is 'myapp.models.NewManager[myapp.models.MyModel]'
         reveal_type(MyModel.objects.get())  # N: Revealed type is 'Any'
         reveal_type(MyModel.objects.manager_only_method())  # N: Revealed type is 'builtins.int'
         reveal_type(MyModel.objects.manager_and_queryset_method())  # N: Revealed type is 'builtins.str'
@@ -83,6 +110,8 @@
         -   path: myapp/models.py
             content: |
                 from django.db import models
+                from mypy_django_plugin.lib import generics
+
                 class ModelBaseManager(models.Manager):
                     def manager_only_method(self) -> int:
                         return 1
@@ -91,13 +120,16 @@
                         return 'hello'
 
                 NewManager = ModelBaseManager.from_queryset(ModelQuerySet, class_name='NewManager')
+                from mypy_django_plugin.lib import generics
+                generics.make_classes_generic(NewManager)
+
                 class MyModel(models.Model):
-                    objects = NewManager()
+                    objects = NewManager['MyModel']()
 
 -   case: from_queryset_with_class_inheritance
     main: |
         from myapp.models import MyModel
-        reveal_type(MyModel().objects)  # N: Revealed type is 'myapp.models.MyModel_NewManager[myapp.models.MyModel]'
+        reveal_type(MyModel().objects)  # N: Revealed type is 'myapp.models.NewManager[myapp.models.MyModel*]'
         reveal_type(MyModel().objects.get())  # N: Revealed type is 'myapp.models.MyModel*'
         reveal_type(MyModel().objects.queryset_method())  # N: Revealed type is 'builtins.str'
     installed_apps:
@@ -115,13 +147,17 @@
                     pass
 
                 NewManager = BaseManager.from_queryset(ModelQuerySet)
+
+                from mypy_django_plugin.lib import generics
+                generics.make_classes_generic(NewManager)
+
                 class MyModel(models.Model):
-                  objects = NewManager()
+                    objects = NewManager['MyModel']()
 
 -   case: from_queryset_with_manager_in_another_directory_and_imports
     main: |
         from myapp.models import MyModel
-        reveal_type(MyModel().objects)  # N: Revealed type is 'myapp.models.MyModel_NewManager[myapp.models.MyModel]'
+        reveal_type(MyModel().objects)  # N: Revealed type is 'myapp.managers.NewManager[myapp.models.MyModel*]'
         reveal_type(MyModel().objects.get())  # N: Revealed type is 'myapp.models.MyModel*'
         reveal_type(MyModel().objects.queryset_method)  # N: Revealed type is 'def (param: Union[builtins.str, None] =) -> Union[builtins.str, None]'
         reveal_type(MyModel().objects.queryset_method('str'))  # N: Revealed type is 'Union[builtins.str, None]'
@@ -134,8 +170,11 @@
                 from django.db import models
                 from myapp.managers import NewManager
 
+                from mypy_django_plugin.lib import generics
+                generics.make_classes_generic(NewManager)
+
                 class MyModel(models.Model):
-                    objects = NewManager()
+                    objects = NewManager['MyModel']()
         -   path: myapp/managers.py
             content: |
                 from typing import Optional
@@ -151,7 +190,7 @@
     disable_cache: true
     main: |
         from myapp.models import MyModel
-        reveal_type(MyModel().objects)  # N: Revealed type is 'myapp.models.MyModel_NewManager[myapp.models.MyModel]'
+        reveal_type(MyModel().objects)  # N: Revealed type is 'myapp.managers.NewManager[myapp.models.MyModel*]'
         reveal_type(MyModel().objects.get())  # N: Revealed type is 'myapp.models.MyModel*'
         reveal_type(MyModel().objects.base_queryset_method)  # N: Revealed type is 'def (param: Union[builtins.int, builtins.str]) -> <nothing>'
         reveal_type(MyModel().objects.base_queryset_method(2))  # N: Revealed type is '<nothing>'
@@ -163,8 +202,12 @@
             content: |
                 from django.db import models
                 from myapp.managers import NewManager
+
+                from mypy_django_plugin.lib import generics
+                generics.make_classes_generic(NewManager)
+
                 class MyModel(models.Model):
-                    objects = NewManager()
+                    objects = NewManager['MyModel']()
         -   path: myapp/managers.py
             content: |
                 from django.db import models
@@ -179,3 +222,59 @@
                 class BaseQuerySet(models.QuerySet):
                     def base_queryset_method(self, param: Union[int, str]) -> NoReturn:
                         raise ValueError
+
+
+-   case: from_queryset_with_inherited_manager_and_fk_to_auth_contrib
+    disable_cache: true
+    main: |
+        from myapp.base_queryset import BaseQuerySet
+        reveal_type(BaseQuerySet().base_queryset_method)  # N: Revealed type is 'def (param: builtins.dict[builtins.str, Union[builtins.int, builtins.str]]) -> Union[builtins.int, builtins.str]'
+
+        from django.contrib.auth.models import Permission
+        reveal_type(Permission().another_models)  # N: Revealed type is 'django.db.models.manager.RelatedManager[myapp.models.AnotherModelInProjectWithContribAuthM2M]'
+
+        from myapp.managers import NewManager
+        reveal_type(NewManager())  # N: Revealed type is 'myapp.managers.NewManager[<nothing>]'
+        reveal_type(NewManager().base_queryset_method)  # N: Revealed type is 'def (param: builtins.dict[builtins.str, Union[builtins.int, builtins.str]]) -> Union[builtins.int, builtins.str]'
+
+        from myapp.models import MyModel
+        reveal_type(MyModel().objects)  # N: Revealed type is 'myapp.models.MyModel_NewManager[myapp.models.MyModel]'
+        reveal_type(MyModel().objects.get())  # N: Revealed type is 'myapp.models.MyModel*'
+        reveal_type(MyModel().objects.base_queryset_method)  # N: Revealed type is 'def (param: builtins.dict[builtins.str, Union[builtins.int, builtins.str]]) -> Union[builtins.int, builtins.str]'
+    installed_apps:
+        - myapp
+        - django.contrib.auth
+    files:
+        -   path: myapp/__init__.py
+        -   path: myapp/models.py
+            content: |
+                from django.db import models
+                from myapp.managers import NewManager
+                from django.contrib.auth.models import Permission
+
+                from mypy_django_plugin.lib import generics
+                generics.make_classes_generic(NewManager)
+
+                class MyModel(models.Model):
+                    objects = NewManager['MyModel']()
+
+                class AnotherModelInProjectWithContribAuthM2M(models.Model):
+                    permissions = models.ForeignKey(
+                        Permission,
+                        on_delete=models.PROTECT,
+                        related_name='another_models'
+                    )
+        -   path: myapp/managers.py
+            content: |
+                from django.db import models
+                from myapp.base_queryset import BaseQuerySet
+                class ModelQuerySet(BaseQuerySet):
+                    pass
+                NewManager = models.Manager.from_queryset(ModelQuerySet)
+        -   path: myapp/base_queryset.py
+            content: |
+                from typing import Union, Dict
+                from django.db import models
+                class BaseQuerySet(models.QuerySet):
+                    def base_queryset_method(self, param: Dict[str, Union[int, str]]) -> Union[int, str]:
+                        return param["hello"]

test-data/typecheck/managers/querysets/test_values_list.yml

@@ -220,6 +220,9 @@
         -   path: myapp/models.py
             content: |
                 from django.db import models
+                from mypy_django_plugin.lib import generics
+                generics.make_classes_generic(models.QuerySet)
+
                 class TransactionQuerySet(models.QuerySet['Transaction']):
                     pass
                 class Transaction(models.Model):

test-data/typecheck/managers/test_managers.yml

@@ -30,6 +30,7 @@
                 class Child(Parent):
                     pass
 
+
 -   case: test_model_objects_attribute_present_in_case_of_model_cls_passed_as_generic_parameter
     main: |
         from myapp.models import Base, MyModel
@@ -55,23 +56,27 @@
                     def method(self) -> None:
                         reveal_type(self.model_cls._default_manager)  # N: Revealed type is 'django.db.models.manager.Manager[myapp.models.MyModel]'
 
+
 -   case: if_custom_manager_defined_it_is_set_to_default_manager
     main: |
         from myapp.models import MyModel
         reveal_type(MyModel._default_manager)  # N: Revealed type is 'myapp.models.CustomManager[myapp.models.MyModel]'
+        reveal_type(MyModel._default_manager.get())  # N: Revealed type is 'myapp.models.MyModel*'
     installed_apps:
         - myapp
     files:
         -   path: myapp/__init__.py
         -   path: myapp/models.py
             content: |
-                from typing import TypeVar
                 from django.db import models
-                _T = TypeVar('_T', bound=models.Model)
+                from mypy_django_plugin.lib import generics
-                class CustomManager(models.Manager[_T]):
+                generics.make_classes_generic(models.Manager)
+
+                class CustomManager(models.Manager['MyModel']):
                     pass
                 class MyModel(models.Model):
-                    manager = CustomManager['MyModel']()
+                    manager = CustomManager()
+
 
 -   case: if_default_manager_name_is_passed_set_default_manager_to_it
     main: |
@@ -83,40 +88,54 @@
         -   path: myapp/__init__.py
         -   path: myapp/models.py
             content: |
-                from typing import TypeVar
                 from django.db import models
 
-                _T = TypeVar('_T', bound=models.Model)
+                from mypy_django_plugin.lib import generics
-                class Manager1(models.Manager[_T]):
+                generics.make_classes_generic(models.Manager)
+
+                class Manager1(models.Manager['MyModel']):
                     pass
-                class Manager2(models.Manager[_T]):
+                class Manager2(models.Manager['MyModel']):
                     pass
                 class MyModel(models.Model):
                     class Meta:
                         default_manager_name = 'm2'
-                    m1 = Manager1['MyModel']()
+                    m1 = Manager1()
-                    m2 = Manager2['MyModel']()
+                    m2 = Manager2()
 
--   case: test_leave_as_is_if_objects_is_set_and_fill_typevars_with_outer_class
+
+-   case: manager_requires_type_annotation_to_be_set_if_generic_is_not_specified
     main: |
-        from myapp.models import MyUser
+        from myapp.models import MyModel
-        reveal_type(MyUser.objects)  # N: Revealed type is 'myapp.models.UserManager[myapp.models.MyUser]'
+        reveal_type(MyModel.objects)  # N: Revealed type is 'myapp.models.MyManager'
-        reveal_type(MyUser.objects.get())  # N: Revealed type is 'myapp.models.MyUser*'
+        reveal_type(MyModel.objects.get())  # N: Revealed type is 'Any'
-        reveal_type(MyUser.objects.get_or_404())  # N: Revealed type is 'myapp.models.MyUser'
+        reveal_type(MyModel.objects2)  # N: Revealed type is 'myapp.models.MyGenericManager[Any]'
+        reveal_type(MyModel.objects2.get())  # N: Revealed type is 'Any'
+        reveal_type(MyModel.objects3)  # N: Revealed type is 'myapp.models.MyGenericManager[myapp.models.MyModel]'
+        reveal_type(MyModel.objects3.get())  # N: Revealed type is 'myapp.models.MyModel*'
     installed_apps:
         - myapp
     files:
         -   path: myapp/__init__.py
         -   path: myapp/models.py
             content: |
+                from typing import TypeVar
                 from django.db import models
 
-                class UserManager(models.Manager['MyUser']):
+                from mypy_django_plugin.lib import generics
-                    def get_or_404(self) -> 'MyUser':
+                generics.make_classes_generic(models.Manager)
+
+                class MyManager(models.Manager):
+                    pass
+                _T = TypeVar('_T', bound=models.Model)
+                class MyGenericManager(models.Manager[_T]):
                     pass
 
-                class MyUser(models.Model):
+                class MyModel(models.Model):
-                    objects = UserManager()
+                    objects = MyManager()
+                    objects2 = MyGenericManager()  # E: Need type annotation for 'objects2'
+                    objects3: 'MyGenericManager[MyModel]' = MyGenericManager()
+
 
 -   case: model_imported_from_different_file
     main: |
@@ -139,13 +158,14 @@
                 class Inventory(models.Model):
                     pass
 
+
 -   case: managers_that_defined_on_other_models_do_not_influence
     main: |
         from myapp.models import AbstractPerson, Book
         reveal_type(AbstractPerson.abstract_persons)  # N: Revealed type is 'django.db.models.manager.Manager[myapp.models.AbstractPerson]'
-        reveal_type(Book.published_objects)  # N: Revealed type is 'myapp.models.PublishedBookManager[myapp.models.Book]'
+        reveal_type(Book.published_objects)  # N: Revealed type is 'myapp.models.PublishedBookManager'
         Book.published_objects.create(title='hello')
-        reveal_type(Book.annotated_objects)  # N: Revealed type is 'myapp.models.AnnotatedBookManager[myapp.models.Book]'
+        reveal_type(Book.annotated_objects)  # N: Revealed type is 'myapp.models.AnnotatedBookManager'
         Book.annotated_objects.create(title='hello')
     installed_apps:
         - myapp
@@ -155,6 +175,9 @@
             content: |
                 from django.db import models
 
+                from mypy_django_plugin.lib import generics
+                generics.make_classes_generic(models.Manager)
+
                 class AbstractPerson(models.Model):
                     abstract_persons = models.Manager['AbstractPerson']()
                 class PublishedBookManager(models.Manager['Book']):
@@ -166,7 +189,8 @@
                     published_objects = PublishedBookManager()
                     annotated_objects = AnnotatedBookManager()
 
--   case: managers_inherited_from_abstract_classes_multiple_inheritance
+
+-   case: managers_inherited_from_abstract_classes_multiple_inheritance_do_not_warn_on_liskov
     main: |
     installed_apps:
         - myapp
@@ -175,6 +199,9 @@
         -   path: myapp/models.py
             content: |
                 from django.db import models
+                from mypy_django_plugin.lib import generics
+                generics.make_classes_generic(models.Manager)
+
                 class CustomManager1(models.Manager['AbstractBase1']):
                     pass
                 class AbstractBase1(models.Model):
@@ -193,6 +220,7 @@
                 class Child(AbstractBase1, AbstractBase2):
                     pass
 
+
 -   case: model_has_a_manager_of_the_same_type
     main: |
         from myapp.models import UnrelatedModel, MyModel
@@ -208,59 +236,21 @@
         -   path: myapp/models.py
             content: |
                 from django.db import models
+                from mypy_django_plugin.lib import generics
+                generics.make_classes_generic(models.Manager)
+
                 class UnrelatedModel(models.Model):
                     objects = models.Manager['UnrelatedModel']()
 
                 class MyModel(models.Model):
                     pass
 
--   case: manager_without_annotation_of_the_model_gets_it_from_outer_one
-    main: |
-        from myapp.models import UnrelatedModel2, MyModel2
-        reveal_type(UnrelatedModel2.objects) # N: Revealed type is 'django.db.models.manager.Manager[myapp.models.UnrelatedModel2]'
-        reveal_type(UnrelatedModel2.objects.first()) # N: Revealed type is 'Union[myapp.models.UnrelatedModel2*, None]'
-
-        reveal_type(MyModel2.objects) # N: Revealed type is 'django.db.models.manager.Manager[myapp.models.MyModel2]'
-        reveal_type(MyModel2.objects.first()) # N: Revealed type is 'Union[myapp.models.MyModel2*, None]'
-    installed_apps:
-        - myapp
-    files:
-        -   path: myapp/__init__.py
-        -   path: myapp/models.py
-            content: |
-                from django.db import models
-                class UnrelatedModel2(models.Model):
-                    objects = models.Manager()
-
-                class MyModel2(models.Model):
-                    pass
-
--   case: inherited_manager_has_the_proper_type_of_model
-    main: |
-        from myapp.models import ParentOfMyModel3, MyModel3
-        reveal_type(ParentOfMyModel3.objects) # N: Revealed type is 'django.db.models.manager.Manager[myapp.models.ParentOfMyModel3]'
-        reveal_type(ParentOfMyModel3.objects.first()) # N: Revealed type is 'Union[myapp.models.ParentOfMyModel3*, None]'
-
-        reveal_type(MyModel3.objects) # N: Revealed type is 'django.db.models.manager.Manager[myapp.models.MyModel3]'
-        reveal_type(MyModel3.objects.first()) # N: Revealed type is 'Union[myapp.models.MyModel3*, None]'
-    installed_apps:
-        - myapp
-    files:
-        -   path: myapp/__init__.py
-        -   path: myapp/models.py
-            content: |
-                from django.db import models
-                class ParentOfMyModel3(models.Model):
-                    objects = models.Manager()
-
-                class MyModel3(ParentOfMyModel3):
-                    pass
 
 -   case: inheritance_with_explicit_type_on_child_manager
     main: |
         from myapp.models import ParentOfMyModel4, MyModel4
-        reveal_type(ParentOfMyModel4.objects) # N: Revealed type is 'django.db.models.manager.Manager[myapp.models.ParentOfMyModel4]'
+        reveal_type(ParentOfMyModel4.objects) # N: Revealed type is 'django.db.models.manager.Manager[Any]'
-        reveal_type(ParentOfMyModel4.objects.first()) # N: Revealed type is 'Union[myapp.models.ParentOfMyModel4*, None]'
+        reveal_type(ParentOfMyModel4.objects.first()) # N: Revealed type is 'Union[Any, None]'
 
         reveal_type(MyModel4.objects) # N: Revealed type is 'django.db.models.manager.Manager[myapp.models.MyModel4]'
         reveal_type(MyModel4.objects.first()) # N: Revealed type is 'Union[myapp.models.MyModel4*, None]'
@@ -271,54 +261,22 @@
         -   path: myapp/models.py
             content: |
                 from django.db import models
+                from mypy_django_plugin.lib import generics
+                generics.make_classes_generic(models.Manager)
+
                 class ParentOfMyModel4(models.Model):
-                    objects = models.Manager()
+                    objects = models.Manager()  # E: Need type annotation for 'objects'
 
                 class MyModel4(ParentOfMyModel4):
                     objects = models.Manager['MyModel4']()
 
-# TODO: make it work someday
-#-   case: inheritance_of_two_models_with_custom_objects_manager
-#    main: |
-#        from myapp.models import MyBaseUser, MyUser
-#        reveal_type(MyBaseUser.objects)  # N: Revealed type is 'myapp.models.MyBaseManager[myapp.models.MyBaseUser]'
-#        reveal_type(MyBaseUser.objects.get())  # N: Revealed type is 'myapp.models.MyBaseUser'
-#
-#        reveal_type(MyUser.objects)  # N: Revealed type is 'myapp.models.MyManager[myapp.models.MyUser]'
-#        reveal_type(MyUser.objects.get())  # N: Revealed type is 'myapp.models.MyUser'
-#    installed_apps:
-#        - myapp
-#    files:
-#        -   path: myapp/__init__.py
-#        -   path: myapp/models.py
-#            content: |
-#                from django.db import models
-#
-#                class MyBaseManager(models.Manager):
-#                    pass
-#                class MyBaseUser(models.Model):
-#                    objects = MyBaseManager()
-#
-#                class MyManager(models.Manager):
-#                    pass
-#                class MyUser(MyBaseUser):
-#                    objects = MyManager()
 
--   case: custom_manager_returns_proper_model_types
+-   case: manager_defined_as_a_nested_class
     main: |
-        from myapp.models import User
+        from myapp.models import MyModel
-        reveal_type(User.objects)  # N: Revealed type is 'myapp.models.User_MyManager2[myapp.models.User]'
+        reveal_type(MyModel.objects)  # N: Revealed type is 'myapp.models.MyModel.MyManager'
-        reveal_type(User.objects.select_related())  # N: Revealed type is 'myapp.models.User_MyManager2[myapp.models.User]'
+        reveal_type(MyModel.objects.get())  # N: Revealed type is 'myapp.models.MyModel*'
-        reveal_type(User.objects.get())  # N: Revealed type is 'myapp.models.User*'
+        reveal_type(MyModel.objects.mymethod())  # N: Revealed type is 'builtins.int'
-        reveal_type(User.objects.get_instance())  # N: Revealed type is 'builtins.int'
-        reveal_type(User.objects.get_instance_untyped('hello'))  # N: Revealed type is 'Any'
-
-        from myapp.models import ChildUser
-        reveal_type(ChildUser.objects)  # N: Revealed type is 'myapp.models.ChildUser_MyManager2[myapp.models.ChildUser]'
-        reveal_type(ChildUser.objects.select_related())  # N: Revealed type is 'myapp.models.ChildUser_MyManager2[myapp.models.ChildUser]'
-        reveal_type(ChildUser.objects.get())  # N: Revealed type is 'myapp.models.ChildUser*'
-        reveal_type(ChildUser.objects.get_instance())  # N: Revealed type is 'builtins.int'
-        reveal_type(ChildUser.objects.get_instance_untyped('hello'))  # N: Revealed type is 'Any'
     installed_apps:
         - myapp
     files:
@@ -326,12 +284,11 @@
         -   path: myapp/models.py
             content: |
                 from django.db import models
-                class MyManager(models.Manager):
+                from mypy_django_plugin.lib import generics
-                    def get_instance(self) -> int:
+                generics.make_classes_generic(models.Manager)
+
+                class MyModel(models.Model):
+                    class MyManager(models.Manager['MyModel']):
+                        def mymethod(self) -> int:
                             pass
-                    def get_instance_untyped(self, name):
-                        pass
-                class User(models.Model):
-                    objects = MyManager()
-                class ChildUser(models.Model):
                     objects = MyManager()

test-data/typecheck/utils/test_functional.yml

@@ -1,18 +0,0 @@
--   case: cached_property_class_vs_instance_attributes
-    main: |
-      from django.utils.functional import cached_property
-      from typing import List
-
-      class Foo:
-          @cached_property
-          def attr(self) -> List[str]: ...
-
-          reveal_type(attr)      # N: Revealed type is 'django.utils.functional.cached_property[builtins.list*[builtins.str]]'
-          reveal_type(attr.name) # N: Revealed type is 'builtins.str'
-
-      reveal_type(Foo.attr)      # N: Revealed type is 'django.utils.functional.cached_property[builtins.list*[builtins.str]]'
-      reveal_type(Foo.attr.func) # N: Revealed type is 'def (*Any, **Any) -> builtins.list*[builtins.str]'
-
-      f = Foo()
-      reveal_type(f.attr)        # N: Revealed type is 'builtins.list*[builtins.str]'
-      f.attr.name                # E: "List[str]" has no attribute "name"

test-output/round-table.gv

@@ -0,0 +1,3 @@
+digraph {
+	FuncDef [label="My FuncDef"]
+}