* Add test case reproducing Sequence name not defined issue
* Resolve all manager methods as attribute
This changes to logic for resolving methods from the base QuerySet class
on managers from copying the methods to use the attribute approach
that's already used for methods from custom querysets. This resolves the
phantom type errors that stem from the copying.
* Disable cache in test case
Make sure the test will fail regardless of which mypy.ini file is being using.
Co-authored-by: Petter Friberg <petter@5monkeys.se>
* Update comments related to copying methods
* Use a predefined list of manager methods to update
The list of manager methods that returns a queryset, and thus need to
have it's return type changed, is small and well defined. Using a
predefined list of methods rather than trying to detect these at runtime
makes the code much more readable and probably faster as well.
Also add `extra()` to the methods tested in
from_queryset_includes_methods_returning_queryset, and sort the methods
alphabetically.
* Revert changes in .github/workflows/tests.yml
With cache_disable: true on the test case this is no longer needed to
reproduce the bug.
* Remove unsued imports and change type of constant
- Remove unused imports left behind
- Change MANAGER_METHODS_RETURNING_QUERYSET to Final[FrozenSet[str]]
* Import Final from typing_extensions
Was added in 3.8, we still support 3.7
* Sort imports properly
* Remove explicit typing of final frozenset
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
* Add comment for test case
* Fix typo
* Rename variable
Co-authored-by: Petter Friberg <petter@5monkeys.se>
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
In addition to str, PostgreSQL cursors accept the
psycopg2.sql.Composable type, which is useful for guarding against SQL
injections when building raw queries that can’t be parameterized in
the normal way (e.g. interpolating identifiers).
In order to avoid reintroducing a dependency on psycopg2, we define a
Protocol that matches psycopg2.sql.Composable.
Documentation: https://www.psycopg.org/docs/sql.html
Related: https://github.com/python/typeshed/pull/7494
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
Char fields with blank=True set should not be considered nullable in the
context of values() and values_list() querysets.
I'm also not a huge fan of the way these fields are made optional in the
constructur to the model classes, it feels like it would be better to
mark the arguments as having a default value, rather than allow sending
in None, but I'd rather keep this fix small and look at the overall
problem at a later point.
If a django model has a Manager class that cannot be resolved statically
(if it is generated in a way where we cannot import it, like `objects =
my_manager_factory()`), we fallback to the default related manager, so
you at least get a base level of working type checking.
* Fix manager types scope
* Restore incremental mode and mention in developer docs
* Separate dev mypy config and regular one
* Document config files usage
- Updates test_model_field_classes_from_existing_locations to account
for the behaviour change in https://github.com/python/mypy/pull/12663
- Bumps the version of django-stubs for a new release
When fetching a related field in a values_list queryset Django will
return the object primary key, not model instances as was previously
what the mypy plugin assumed.
* Bump mypy to 0.95x
* Remove the * for inferred types
There was an upstream change (https://github.com/python/mypy/pull/12459)
to remove * from inferred types in the reveal_type output.
As we are asserting the * to exist, all the test cases are now failing
on the 0.950 release. Removing the expected * to mirror the upstream
behaviour change should resolve the test failures.
* Fix stubs related to `(Async)RequestFactory` and `(Async)Client`
* Revert incorrect removal.
* Allow set as `unique_together`, use shared type alias.
* Revert `Q.__init__` to use only `*args, **kwargs` to remove false-positive with `Q(**{...})`
* Add abstract methods to `HttpResponseBase` to create common interface.
* Remove monkey-patched attributes from `HttpResponseBase` subclasses.
* Add QueryDict mutability checks (+ plugin support)
* Fix lint
* Return back GenericForeignKey to `Options.get_fields`
* Minor fixup
* Make plugin code typecheck with `--warn-unreachable`, minor performance increase.
* Better types for `{unique, index}_together` and Options.
* Fix odd type of `URLResolver.urlconf_name` which isn't a str actually.
* Better types for field migration operations.
* Revert form.files to `MultiValueDict[str, UploadedFile]`
* Compatibility fix (#916)
* Do not assume that `Annotated` is always related to django-stubs (fixes#893)
* Restrict `FormView.get_form` return type to `_FormT` (class type argument). Now it is resolved to `form_class` argument if present, but also errors if it is not subclass of _FormT
* Fix CI (make test runnable on 3.8)
* Fix CI (make test runnable on 3.8 _again_)
* Make module declaration precise.
* Make settings match real file.
* Replace `include` with import.
* Make types more specific.
* Replace `WSGIRequest` with `HttpRequest` where possible.
* Replace all `OrderedDict` occurrences with plain `Dict` (it is not used in Django 3.2 and later)
* Add fake datastructures for convenience: _PropertyDescriptor and _ListOrTuple now can live here. Added _IndexableCollection (often useful as alias for 'sequence or queryset')
* Actualize other datastructures.
* Rework MultiValueDict to reflect the fact that some methods can return empty list instead of value.
* Deprecate SafeText in favor of SafeString.
* Minor improvements to utils
* Disallow using str in TimeFormat and DateFormat, drop removed fmt `B`
* Do not let classproperty expect classmethod, make return value covariant.
* Sync with real file.
* Improve types for timezone.
* Sync deprecated, new and removed features in translation utils.
* Drop removed files, sync huge deprecations.
* Fix incompatible decorators (properties, contextmanagers)
* Rework pagination.
* Sync validators with real code. Add _ValidatorCallable for any external use (field validation etc.)
* Add shared type definitions (for fields of both forms and models). Actualize model fields. Mark keyword-only args explicitly in stubs (where code uses **kwargs). Disallow bytes for verbose_name.
* Make all checks return Sequence[CheckMessage] or subclass to be covariant.
* Add bidirectional references between backend.base and other files. Replace some Any's with specific types.
* Actualize db.migrations: remove removed methods, replace "None" annotations in wrong places, improve some wrong annotations.
* Actualize db.utils to match real file.
* Replace FileResponse and TemplateResponse with HttpResponse(Base) where needed: at least HttpResponseNotModified/HttpResponseRedirect can be returned instead of it, so annotation was wrong.
* Replace Any in forms where possible. Actualize class bases and method arguments.
* Improve typing of serializers.
* Actualize views, rename variable bound to Model to _M for consistency.
* Make types of file-related code consistent. Disallow using bytes as path, because many methods expect str-only paths. Make File inherit from IO[AnyStr] instead of IO[Any]: it makes impossible to instantiate file of union type, but allows precise types for some methods.
* Minor improvements: stop using None as annotation in wrong places, replace obvious Any's with precise types, actualize methods (missing/renamed/signature changed).
* Allow less specific containers, replace Any's with specific types.
* Improve types for requests and responses.
* Use AbstractBaseUser instead of User in auth.
* Use broader type for permission_required
* Use wider container types. Add 'type: ignore' to avoid issues with mypy.stubtest.
* Disallow using backend class as argument (it is passed to import_string).
* Add required methods to PasseordValidator.
* Allow using Path instance as argument.
* Actualize methods.
* Add 'type: ignore' to avoid issues with mypy.stubtest.
* Replace Any's with specific types and BaseForm with ModelForm.
* Actualize contrib.postgres
* Remove render_to_response, add 'get_absolute_url' to corresponding protocol.
* Actualize signers.
* Use precise types for handlers. Disallow str as stream type for LimitedStream.
* Exact types for ValidationError
* Replace wrong used Union with Sequence.
* Actualize static handlers.
* More specific types for admin. Fixes#874.
* Improve types and replace 'Tags' with str (it isn't Enum, so annotation was wrong).
* Replace Any with specific types, actualize signatures.
* Add async variants of handlers and clients. Use fake class to distinguish between request types in RequestFactory and AsyncRequestFactory.
* Fix signature, minor improvements.
* Actualize signatures and class names, replace Any with more specific types.
* Fix signature.
* Add some missing methods to Collector
* Combinable rarely returns Self type: almost always it's CombinedExpression.
* No Random in source anymore.
* Drop removed SimpleCol.
* Replace _OutputField with Field: nothing in docs about strings.
* Introduce reusable types, add missing methods. Remove strange types (probably created by stubgen). Remove RawQuery from Compiler: it obviously doesn't work with RawQuery.
* Use literal constants.
* Actualize base classes.
* Callable is not accepted by get_field.
* Add precise types.
* Use property and broader containers where possible. Add missing methods.
* Actualize indexes.
* More specific types for signals.
* Fix signatures, drop missing methods.
* Actualize window functions to match source.
* Actualize text functions, add missing methods, use type aliases for consistency.
* Add missing property decorators, methods and attributes. Use type aliases. Remove absent YearComparisonLookup and any SafeText references (they aren't related to lookups at all).
* Use bound TypeVar, mark all BuiltinLookup descendants as generic explicitly. Remove strange Union from Lookup.__init__
* Apply type alias, fix base class and argument name.
* Actualize BaseExpression methods.
* Fix imports.
* Add missing class and fix incompatible bases.
* Use same types in __init__ and attribute.
* OrderBy accepts F or Expression.
* Non-expressions are converted to Values.
* Add missing attributes.
* Add missing methods, fix 'None' argument type.
* Define properties where possible, remove 'None' argument annotations, remove inadequate type in make_immutable_fields_list.
* Remove absent QueryWrapper. Replace some Any with precise types.
* Fix wrong types and actualize signatures. Deny ManagerDescriptor.__get__ on model instances.
* Use more specific types.
* Arity can be None in subclasses.
* Reformat with black
* Make DeletionMixin generic.
* Fix wrong type variable in _RequestFactory.
* Fix variable name in signature.
* Disallow returning None from Form.clean()
* Allow again returning None from Form.clean
* Drop some unused imports.
* Add tests for MultiValueDict.
* Add tests for utils.timezone.
* Fix#834.
* Add more files to import_all test
* Allow None for `context_object_name`
* Fix CI
* Fix test to work on python 3.8
* Add reproducer for failing case
* Emit warning instead of crashing when encountering enum
* Remove prints, slightly tweak error message
* Remove unused import
* Run black and isort
* Run isort on .pyi file
* Remove unrelated issue from test case
* Fix missing related managers on some models
I was seeing an issue where some related managers were missing from some
models. Traced the issue down to this line, where it appears that if we
hit a relation with a non-default(?) reverse manager the iteration
stopped. I _think_ this is supposed to be a continue statement instead.
It appears to work in the project I'm working in at least.
* Add test case
* Add test case
* DatabaseClient: add missing class method `settings_to_cmd_args_env` and add missing arg to `runshell`
The `DatabaseClient.runshell()` was refactored in Django 3.2 to be more standardized
across the different database backends.
* DatabaseClient: make `settings_dict` dict type better by specifying key/value types
* tests: add missing db backend client files to `import_all_modules`
* _default_manager has more specific type for TypeVars
* remove unnecessary # type: ignore
* add test for _base_manager
* add overloads for classproperty.__get__
* readd # type: ignore for WSGIRequestHandler.connection, fails in github's pipeline
* fix _base_manager test: mypy reveals an inferred type
Co-authored-by: Michael Pöhle <michael.poehle@polyteia.de>
* Refactor to more easily support additional config options
* Notify when Manager.from_queryset happens inside model class body
- A warning will be emitted whenever `Manager.from_queryset` happens
inside of a model class body
* Resolve generated default manager types before final iteration
A default manager on a model should always exist, eventually. Although,
we extend to look through dynamically generated managers on each
iteration instead of deferring until the final iteration.
Instead of copying methods over from a QuerySet passed to a basemanager
when invoking '<BaseManager>.from_queryset', any QuerySet methods are
declared as attributes on the manager.
This allows us to properly lookup any QuerySet method types via a
'get_attribute_hook' and will thus remove disorienting phantom errors
occuring from mypy trying to resolve types only existing in the module
where the _original_ (and real) queryset method was declared.
* Add failing test for relation to model inheriting `objects`
Fails with:
```
pytest_mypy_plugins.utils.TypecheckAssertionError: Invalid output:
Expected:
main:2: note: Revealed type is "myapp.models.MyUser*"
main:3: note: Revealed type is "myapp.models.MyUser*"
<45 (diff)
<45 (diff)
Actual:
main:2: note: Revealed type is "myapp.models.MyUser*"
main:3: note: Revealed type is "myapp.models.MyUser*"
main:6: error: "MyUser" has no attribute "book_set" (diff)
main:6: note: Revealed type is "Any" (diff)
main:7: error: "MyUser" has no attribute "article_set" (diff)
main:7: note: Revealed type is "Any" (diff)
```
* Make AddRelatedManagers look for "objects" on parent model
Previously, AddRelatedManagers would fail if a related model had inherited
its `objects` field from a parent class. This would result in missing
relation attributes. This is fixed by using `get()` instead of `names`;
the former searches the MRO for the symbol, whereas the latter only looks
for symbols declared directly on the class.
* Fix `MyModel.objects.filter(...).my_method()`
* Fix regression: `MyModel.objects.filter(...).my_method()` no longer worked when using from_queryset
This also fixes the self-type of the copied-over methods of the manager generated by from_queryset.
Previously it was not parameterized by the model class, but used Any.
The handling of unbound types is not tested here as I have not been able to
find a way to create a test case for it. It has been manually tested
against an internal codebase.
* Remove unneeded defer.
* Add datetime to set_type of DateTimeField
`DateTimeField` was missing `datetime` as a valid set type. But Django clearly accepts `datetime`.
* Fix test for DateTimeField type change
datetime is now a valid set type for DateTimeField
* QuerySet.annotate returns self-type. Attribute access falls back to Any.
- QuerySets that have an annotated model do not report errors during .filter() when called with invalid fields.
- QuerySets that have an annotated model return ordinary dict rather than TypedDict for .values()
- QuerySets that have an annotated model return Any rather than typed Tuple for .values_list()
* Fix .annotate so it reuses existing annotated types. Fixes error in typechecking Django testsuite.
* Fix self-typecheck error
* Fix flake8
* Fix case of .values/.values_list before .annotate.
* Extra ignores for Django 2.2 tests (false positives due to tests assuming QuerySet.first() won't return None)
Fix mypy self-check.
* More tests + more precise typing in case annotate called before values_list.
Cleanup tests.
* Test and fix annotate in combination with values/values_list with no params.
* Remove line that does nothing :)
* Formatting fixes
* Address code review
* Fix quoting in tests after mypy changed things
* Use Final
* Use typing_extensions.Final
* Fixes after ValuesQuerySet -> _ValuesQuerySet refactor. Still not passing tests yet.
* Fix inheritance of _ValuesQuerySet and remove unneeded type ignores.
This allows the test
"annotate_values_or_values_list_before_or_after_annotate_broadens_type"
to pass.
* Make it possible to annotate user code with "annotated models", using PEP 583 Annotated type.
* Add docs
* Make QuerySet[_T] an external alias to _QuerySet[_T, _T].
This currently has the drawback that error messages display the internal type _QuerySet, with both type arguments.
See also discussion on #661 and #608.
Fixes#635: QuerySet methods on Managers (like .all()) now return QuerySets rather than Managers.
Address code review by @sobolevn.
* Support passing TypedDicts to WithAnnotations
* Add an example of an error to README regarding WithAnnotations + TypedDict.
* Fix runtime behavior of ValuesQuerySet alias (you can't extend Any, for example).
Fix some edge case with from_queryset after QuerySet changed to be an
alias to _QuerySet. Can't make a minimal test case as this only occurred
on a large internal codebase.
* Fix issue when using from_queryset in some cases when having an argument with a type annotation on the QuerySet.
The mypy docstring on anal_type says not to call defer() after it.
Unlike `List`, which is invariant, `Sequence` is covariant, which lets
`path` accept lists of subsets of the `Union` as well.
I believe this is safe, as django doesn't mutate this input. I found
[this
comment](https://github.com/python/mypy/issues/3351#issuecomment-300447832)
helpful
