Tuya PJ-1203A MatSeePlus Energy Meter (TS0601 _TZE204_81yrt3lo/_TZE284_81yrt3lo) v2 quirk

[jeverley]

Proposed change

Adds support for MatSeePlus PJ-1203A 2 CT clamp Energy Meter.

Addresses Device Support Requests

• [Device Support Request] _TZE204_81yrt3lo TS0601 Zigbee power meter with clamp #2549
• [Device Support Request] TS0601 _TZE284_81yrt3lo Tuya Smart Life Dual channel energy meter #3658

Simplifies original pull requests

• ts0601_energy_meter quirk for Tuya power meter devices (v1 quirk) #2961
• ts0601_energy_meter quirk for Tuya power meter devices (v2 quirk) #3824

Separated from other Tuya devices to isolate logic handling the late flow reporting behaviour summarised below:

There is a known issue on the _TZE204_81yrt3lo (with appVersion 74, stackVersion 0 and hwVersion 1).
The energy_flow datapoints are (incorrectly) emitted during the next update. This is quite problematic
because that means that the direction can be inverted for up to update_frequency seconds.

More detail in the Zigbee2MQTT tuya.ts device handler

The quirk makes use of common power value signing logic for both device variations (combining power and energy flow DP values), however the mitigation for the late energy flow bug is disabled by default.

This is because it is not required in the _TZE284 device revision (sharing the same data points, but reporting the correct direction in the current interval), nor is it required if the _TZE204 device is used to measure power that always flows in the same direction.

Sensor Entities

Electrical Measurement

• Power A
• Power B
• Total Power
• Current A
• Current B
• Power Factor A
• Power Factor B
• Voltage (measured from device power source)
• AC Frequency (measured from device power source)

Metering

• Current Summation Delivered A
• Current Summation Delivered B
• Current Summation Received A
• Current Summation Received B

Configuration Entities

• Reporting interval (10-60 seconds)
• Calibrate reported values by 0-200%, adjustment occurs on the device and all calibration entities are disabled by default.
  • AC Frequency
  • Voltage
  • Summation Delivered (A and B)
  • Summation Received (A and B)
  • Power (A and B)
  • Current (A and B)

Diagnostic Entities

• Late energy flow bug mitigation (A & B), entities enabled by default on _TZE204.

Additional information

• Users of the original draft v1/v2 quirks will need to either manually delete deprecated entities or re-add devices to remove them.

Checklist

• The changes are tested and work correctly
• pre-commit checks pass / the code has been formatted using Black
• Tests have been added to verify that the new code works
• Device diagnostics data has been attached: zha-ae94357e53af2d13d0ee1c425ddd864e-_TZE204_81yrt3lo TS0601-bc41e64265aaa00cab79028802fb739b.json

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.35%. Comparing base (1e5953b) to head (3bea731).
⚠️ Report is 8 commits behind head on dev.

Additional details and impacted files

@@            Coverage Diff             @@
##              dev    #4498      +/-   ##
==========================================
+ Coverage   92.24%   92.35%   +0.11%     
==========================================
  Files         369      372       +3     
  Lines       12094    12274     +180     
==========================================
+ Hits        11156    11336     +180     
  Misses        938      938              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[jeverley]

Quirk itself validated working in all scenarios with the last commit - will update the validate the tests cases tomorrow prior to making a final commit (haven't got access to my dev container today to automate testing these).

[jeverley]

Ready for review, let me know of any issues👍

[YarekTyshchenko]

For me this adds metrics three times, only the suffix _3 actually has data. For some it makes sense to have two, but most of them have three. It's a little confusing what's going on. Seems to work better than my previous quirk file, so I guess its progress.
My device is one of these from ali express:

[jeverley]

Hi @YarekTyshchenko , if you delete and re-pair the device it should remove the redundant sensors.

I simplified a lot of things from that original quirk to make things easier to support and review.

Let me know if that resolves things for you, or if you still have issues.

[YarekTyshchenko]

After recreating its a little better, but the names still don't make sense:

I would expect every duplicate field labelled A or B?
Why is voltage and frequency reported three times, the sensor only has one reference for that

[jeverley]

Thanks for checking @YarekTyshchenko , I'll try and replicate the issue you're seeing with the extra sensors remaining this morning.

The naming of different sensors should be addressed by zigpy/zha#525 once it is merged (it would number them Power 1, Power 2, Power 3 etc).

[therealcmj]

Adding additional data point for you in case it's useful.

I have seemingly the same device as YarekTyshchenko - labeled "Model PJ-1203A" with a marked measuring range of 0.2-80A.

I paired it and it appeared with no entities. I then installed the quirk and hit Reconfigure and got the 1, 2, or 3 of each of the expected sensors. Some of which are reporting numbers and some saying "Unknown".

I then removed the device entirely and added it back and got this:

The difference between the two screen shots is that one of each of Current, Power, and Power Factor disappeared.

[jeverley]

Hi @YarekTyshchenko, @therealcmj thank you both for testing the quirk, I've just pushed 6e44bbb which declares those redundant entities as explicitly unsupported.

This should prevent their creation, let me know if this resolves the issue you both had with the previous version.
If you re-add the device it should clear these up automatically, but you should also now have the option to delete these after updating the quirk.

I've also switched to using the Total power attribute enabled in the recent pull request zigpy/zha#458 for reporting total power.

I've attached a screenshot below from my local instance.

[ksihota]

Edit: switched my clamp leads and got +- all fixed. Biggest issue is guessing which is A and B when adding directly to the energy page. When I create a helper I can see the difference because one is listed with no number and B is marked with _1.
A and B (as marked on the unit) would be more consistent. In the Energy Page I only see Power Power etc. not sure how to differentiate. I am assuming the order: Power (A) 1st and Power (B) 2nd?

Mine's a real mess. I started having problems at the beginning of the new year, I think it was after an update and my sensors started dropping. I figured it had something to do with the quirk so I began hunting down what might have been the problem and found that some of the custom quirk files were supposed to be in the main device handler but it looked like your new one hadn't been added yet. I removed the old quirks and added the new one and after dropping and repairing I got what you can see here. Just in case I also tried using no custom quirk to see if any had been added and got nothing after dropping and repairing so I went back with your latest. I was able to get most of my sensors showing although I cheat and use a single clamp on a single 110 on a double breaker and then multiply by 2 to get an approximation. It works okay unless the device at the other end uses both 220 and 110 like my Oven/microwave combo. Anyways I just build a helper to do that for me. That's the LR Heat Pump in the image. But I am running into issues trying to add items to the energy page because I can't differentiate between A and B. I am also seeing some of the values that used to be positive are now negative and vise versa. Not sure why. I can switch the clamps around or my leads but it just seems strange to me.
Is there any way to have a software toggle built in, if only single direction is needed and then only display power and energy delivered? Also the vocabulary of Power and Summation Delivered/Received does not seem very consistent to me. Why wouldn't it be Power and Energy?
I also want to thank you so much for doing all this work to make these devices work in HA.
PS. How can I tell when this latest quirk will be added to the main system so I don't need to have the custom quirk in my config? I tend to get lost trying to navigate these github pages.

[therealcmj]

@jeverley FWIW here's a screen shot after update and remove/re-add

If you need any logs or other data please let me know.

[jeverley]

Thanks for checking both.

@ksihota I'm glad switching the leads fixed the +/- issue on the power readings, that confirms there isn't an issue with the signing logic.

I can look at adding toggles for this, though I'm very mindful that we should keep quirk complexity to a minimum (It's taken long enough to add support for this device!).

The Summation Received and Summation Delivered terminology is the standard term used for ZigBee energy metering attributes (i.e. sum total energy received / delivered).

I'd rather tackle the attribute naming in a standard way (outlined in zigpy/zha#525), you're correct for now that B CT can be identified as the entity IDs with the _1 suffix.

@therealcmj I note you're using the TZE284 variant, interestingly the summation received and total power attributes aren't showing up (though I can see we have an improvement of the other unsupported entities not being created).

Would it be possible to share the device debug information with and without the custom quirk loaded?

[ksihota]

@jeverley I found that with the TZE204 devices that I have, I could switch the +- by either

1. flipping the clamp direction
2. swapping the leads for the clamp into the device
3. powering the device from the opposite phase
   1 and 2 allows individual control of the AB lines whereas 3 switches both

I can't remember for sure but I believe that originally when I set them up they only displayed Power and Summation (A, B and Total) and the +- indicated direction. Its been a few years so I don't really recall clearly and I am not sure what custom quirk I used originally to get anything to show. The biggest issue I have is the naming problem (knowing which is A and B) so I originally started renaming myself which can become a nightmare if you don't know what is really happening and you don't think it through. In a few cases I ended up getting the Power A and Energy B instead of AA and BB etc.
The other issue is the switching of +- which actually required physical changes rather than just a software switch which allows corrections on the system which can be done remotely.

I do have a couple of questions that are related.

1. Can a device have 2 different custom quirks? For example can a software switch be added as a separate custom quirk that a user can add but would not be needed to be be integrated into the main quirk?
2. When a quirk is added and a device is updated how does the naming work? Does the quirk provide the basic naming to the entities eg Power A; Power B etc?
   Does the new quirk reset the names for a device or do existing names still apply? In other words if I had made changes to the naming conventions (switched an A for a B when I shouldn't have) would this be corrected or would I need to manually correct this? I thought that if I deleted a device and then re-paired it, I would be starting fresh but that does not seem to be the case (or maybe I'm just not doing it properly.) I have a feeling that the device needs some other command to do a factory refresh and I can't find any software command that does that.

[YarekTyshchenko]

Just to weigh in here: I don’t think it’s quirks job to offer any debugging, or swapping direction functionality.
though proper naming would be good.

Note that I have noticed on high load the power reading goes negative briefly before registering correct value

[ksihota]

I agree that quirks should not have to be doing the job of patching poor hardware/software but in the end that is what they do. As a user I would like my hardware to just work but lets face it, that is not going to happen.

As a user I want something that will work easily and if it means I have to build a fix that will save me work than that's what I will do. My problem is that I am just barely grasping the concept of how these quirks work and the syntax used to write them.
I saw that there is the ability to add a software switch in a quirk (the 204 Late Energy flag) that can control the software. If I can do that to control the flow direction, and it makes the device easier to use, then it would be nice to have. If nobody else wants it, but I can add it as something custom for my own setup I would do that.
I guess, in a way, how is that different than adding a custom quirk just to make some of these devices function in HA.
I have basic programming skills but these quirks are something I have not programmed before so that is why I am asking these basic questions.
Are these things possible? I don't know, that's why I posed the questions.
Should they be implemented in a quirk? That depends on what impact it has on the main system and how, and if, it will impact other users.
Should I be able to customize my own setup? Personally, I think yes. That's one of the reasons I am using HA and not relying on proprietary systems.
I am not asking @jeverley to actually change the quirk to do these things but am asking for information and clarity from someone familiar with the systems.

[YarekTyshchenko]

[...] I guess, in a way, how is that different than adding a custom quirk just to make some of these devices function in HA. I have basic programming skills but these quirks are something I have not programmed before so that is why I am asking these basic questions.

I don't think its up to each quirk to provide this. If this is a common scenario, HA itself would allow reversing of sensor values, so that all devices get this functionality.

[...] Are these things possible? I don't know, that's why I posed the questions. [...]

Of course its possible: Copy the quirk file into your own quirk dir, and edit it to your hearts content. It should override these quirks. You can start by duplicating one of the existing boolean setting switches, renaming it, then wiring it up in the code to reverse the polarity