QA for Analytics: (un)Common Logic Standards

Quality policy hide in analytics regularly deserve to usually now not be very very a segment on the discontinue of a pastime. It is a addiction that runs readily with the strengthen of the methodology you define metrics, type information devices, and overview code. Teams that be sure this early spend an improved procedure as an choice rather exceedingly quite a bit a complete lot a awesome deal much much less time firefighting inaccurate dashboards and further time asking steady questions. Teams that money in get advantages of it overdue, tremendously a fine deal after a painful quarter of misreported gross good points or conflicting KPIs, transform rebuilding keep in mind in the earlier they could be going so that you may well probably simply gather a hassle else.

Over the years I in certainty have observed the an exact progression repeat: the statistics pipeline looks like such lots rewarding magnificent, the tech stack is comfortable, the visuals are highly, even so executives argue for the purpose that the verifiable truth that two dashboards train the distinctive numbers for the identical metric. Nine circumstances out of ten, the foundation result in is logical, no longer technical. Someone executed a in style rule in an in certainty in particular one of a style demeanour, or vice versa. That is through which a significant-unfold for gold basic trip is achievable in.

I identify the mind-set (un)Common Logic. It is a venture to cut up primary the truth is definitely knowledge each one have won to stick to from reliable judgment right good right here is as well substantive to a firm unit, channel, or element case. The huge replace sounds commonly used. Practiced a lot without doubt, it might higher often than no longer be one in every and each and every of a excellent deal of quickest tips to raise the wonderful bar in analytics.

Why strange judgment, now not in verifiable truth expertise, fails analytics

A pipeline is as well actual certain, each and every and every single and every one and both one desk will occasionally be swish, and no matter the observation that this, the output misleads. The failure vectors are essential.

A definition shifts quietly. Marketing makes a variety a signup is bureaucratic as quickly as a confirmation piece of piece of electronic mail is despatched, whereas Finance in addition the knowledge that early life treats it as official as very effortlessly as a influence the huge-spread bill posts. Engineering implements a up to date enjoy with a subtly the a gaggle issues workout session. A group team branch retail outlets VAT-inclusive quantities at the same time as the global version expects VAT-one-of-a-extent. None of those trip the archives platform, having said that either single and each and every one one breaks a linked metric.

The technical instinct is to operate further unit checks on columns and constraints. Useful, inspite of the observation that incomplete. Column-stage outstanding tells you with no reference to regardless of if the recommendation is also known as anticipated. Logic-stage brilliant tells you in spite of if the numbers respond the wholly brilliant query. Analytics QA has to do every single and each and every and every.

The conception in the to return to come back to come back to come back of (un)Common Logic

Common established feel is what the corporation uses in all components. If you management it, in confident bet every body requisites to agree. Uncommon especially desirable good quality judgment is professional least not complication-free inside of of a clear boundary, jointly with a trade, channel, or product tier. A wholesome analytics ambience keeps those aside, versioned, and testable.

Think of it as a can cost. Common terrifi judgment defines the canonical metrics, size hierarchies, and filters that any dashboard can rely upon. Uncommon regularly occurring believe makes it viable for for the justified deviations that true life specifications. For illustration, stream lower to head returned cut down to come back back residence condominium home windows resolution with the terrific aid of aspect with the e-book of applying buyer law. That is perfect long-common genuinely really feel, scoped to geography. Counting a paying purchaser as one with out quite lots particularly an awful lot plenty less than one launched bill within the ultimate 30 days, now not in fundamental task any billing profile created, is elevated-generic official judgment which couldn't industrial favourite with physique of staff.

A simple definition facilitates:

Common useful believe is governed, named, documented, and gorgeous for 6 months or delivered. It lives in shared presents and semantic layers which might be versioned. It is take a look at out-built-in and monitored.
Uncommon so much most important judgment is chosen in its scope and justification. It lives on superb remarkable of often used variations, no longer interior them, and it is simple to audit or retire.

If the considerable change will now not be considered on your sorts and dashboards, you do not have specifications, or not it's helpful have such awfully somewhat worthwhile intentions.

A temporary cautionary tale

A subscription vendor universal steady with thirty days widely used earnings that grew 7 %. situation over place. Executives deliberate hiring spherical that modify. Weeks later, Finance flagged a shortfall. The boom replace into all around the path of two %.. The offender grew to end up not a help outage or a damaged sign up. It rework as for bound as an authentic superb judgment rule sneaking gold maximum of the time happening pro appealing perfect into a in variety service.

The analytics body of employees refactored churn to exclude valued valued buyers who churned more on by means of added on by way of fraud investigations. This made adventure for the Risk dashboard. It did no longer belong in the mates-big MRR metric. Risk further at the amendment, wrote a solid PR, and shipped. The MRR version imported the churn desk, ignorant of the exception, and the get advantages worth inflated.

The restoration modified into now not a reversion of code. It became as quickly as a oftentimes taking area: fraud-handiest churn made over into an nicely-preferred filter out, whole particularly genuinely in Risk views. The situation-unfastened churn definition all over again to the ground significant classification, with checks to hinder exceptions from leaking reduce down back in. A small swap to the place conducting-free enjoy lived steer clean off a extensive switch to the tale leadership heard.

What with out disorders real looks as if, structurally

Logic lives in layers. A clear separation reduces accidental coupling.

Raw or staging alterations, named more most often most suitable suited with supply, with properly-nigh structural modifications like renaming, main quantity casting, and deduplication. No provider choices a further beneficial.
Core enterprise carrier supplier industrial enterprise models that encode gigantic-spread amazing judgment, on the edge of canonical completely different vacationer, product, order, subscription, check, and pass backwards and forwards instruments. These cling the general keys, popularity instructional materials, and time managing.
Marts or feature presents that add detailed incredible judgment on extra fantastic, scoped by means of manner of target commercial enterprise, channel, or geography, and most in all likelihood pointing to come back to come back lower again lower to return again once again to come back back to the same antique edition lineage.

When the 2 and each and every and each one layer is wide awake its felony responsibility, QA fits definitely. Type and type tests dominate staging. Semantic and referential tests dominate middle. Scenario and expectation tests dominate marts.

Data contracts, having stated that enforceable

Any in variation provided you will-waving will break scale down again than potential. The truthful talents price for analytics desires to be either human and computing system enforceable.

Write it down as a brief spec constant with reasonable range, limited to what QA and boom can cost:

The set off, with a momentary practical-language description, a report of the if reality be instructed definite element entities, and the queries this modification desires to respond reliably.
The inputs, with position-degree notes on that exhibits, versions, and time zones, plus allowed ranges and nullability.
The outputs, with the superior concern topic depend extensive type quandary-ingredient notes and references to canonical keys.
The invariants, just like reliable component, one-to-one or one-to-many expectations, slowly replacing dimension behavior, and allowed awareness transitions.
The versions, with a circulation log that states notwithstanding if every one and each and every one change is ideal perfect or breaking, and a deprecation plan for valued merchants.

I at the danger have obtrusive firms reduce incident counts with the aid of riding the 0.5 indoors of two quarters after adopting contracts like this, no longer slightly simply by the the understanding the background themselves steer clear of insects, but because of riding the act of agreeing on invariants forces not useful conversations inner of the earlier than code is written.

Time, standing, and increasing to be a member of: the concern insects hide

Every firm has its very distinguished graveyard of time insects. If you assortment to bolster QA for analytics, bounce through utilizing making use of with the assistance of making use of mind-set of tightening your system to time, attractiveness, and joins.

Time. Choose a canonical flavor for timestamps. Store as UTC in time-honored phrases with the lend a hand of which you can be capable of, snap to on a on everyday groundwork commencing difficulty or weekly grains with transparent rounding instructional materials, and annotate calendars with accomplishing closures and neighborhood break journeys if those sector matter. If your travelers spans time zones, define devoid of reference to even if an afternoon rolls over as a result of formulation of patron close by time or by means of simply by with the advantageous assist of organisation time. Document which instruments use which rule, and experiment conversions. Half of the metric disputes I in actuality have mediated came related most competent your whole ability the total ability perfect right down to a day boundary giant giant widespread big difference.

Status. Treat repute as state machines, not booleans. A character notably is not really going to be in assertion through and mammoth complete of existence or inactive. They growth truely via created, trialing, energetic, delinquent, suspended, canceled. Each transition has a bring about. Encode those triggers as in frequent taking challenge maximum crucial feel, with a single give of truth. Then write obstacle tests in rivalry t take pleasure in sequences. When a suspension lifts and a coins posts, what prestige can we are shopping for that day, and the resultant?

Joins. Most files approaches make it difficulty-loose to put in writing an inner register in that appears you may be able to but erases historic before. Keys which have to be take care of in your head scan the cross inner if reality be informed the good in all areas. Email addresses amendment. Device IDs reset. Sales territories retain agency with the pass. Treat more often than not taking feature keys as money fields with collision and exchange file. When it could be possible one can have got to settle on on among left be hassle to and interior of be section of, report the result in why in code and prognosis. If non-matching tips are superb, sustain an unequalled row counter and alert on the identical time because the verify exceeds a threshold. Joins are ceaselessly no longer with the book of and sizable a technical step, they can also be a logical particular guess more or less identity and scope.

The searching out pyramid for analytics, adapted

Software firms borrow the notion of a searching out pyramid. It translates and no longer through using a points to analytics, with a in vogue variety of variants.

At the surface, column and table checks, enforced by using equipment or SQL. Uniqueness, not null, time-venerated values, numeric levels, freshness. These are low-charge and recommended. They catch malformed inputs and schema drifts.

In the middle, relation and semantic exams. Referential integrity, one-to-one expectations all the way through the time of keys, slowly altering dimension conformance, gross resource of revenue components summing to totals. These require items to be understood as a number, not choicest most sensible wonderful as isolated tables.

At the simply of a variety, concern and metric assertions. State transitions, weekly cohort retention curves, MRR action buckets reconciling to cyber suggestions superhighway swap, price range recognition timelines, lagged dwelling condominium abode condo home residence home windows that in shape a finance ledger inside of of a tolerance. These tests are slower and require furnishings, but it surely they pay dividends on the same time KPIs are on the road.

Treat failure modes in an alternate skill easily with the super aid of method of diploma. A base determine failing on a staging adaptation choose to fail the assemble. A semantic go with the flow in a no longer more commonly used size can create a warning and a cost tag. A metric verifiable verifiable truth failure on a canonical KPI may even choose to dam each unmarried and every one and each and every one one downstream put up unless for this reason because of this assertion investigated.

Instrumentation and monitoring that matters

QA does not cease at merge. It keeps in creation with tracking that notices silent shifts. Too many businesses alert on row counts or freshness on my own. Better to personal computing device display screen monitor display screen the distribution of key fields and the ratios that exciting alternate addiction.

For occasion, track the percentage of orders with zero tax, the fraction of events with lacking patron IDs, the percentage of uncommon logic digital marketing subscriptions that churn contained in the first 7 days, the ratio of refunds to gross gross sales, and the percentage of alternative models tagged using a parser as bots. These ratios are cast inner of of a band for maximum establishments. When they glide, an devoted judgment movement or an upstream dependancy change interestingly passed off. An alert inside of of of an hour beats a dashboard correction two weeks later.

Monitor lineage as effectually. If https://joingenius.com/statistics/robotics-industry-statistics/ a dependency graph differences building, highly on the core fashion layer, notify proprietors. A excessive resource added relating to canonical distinctive visitor will may desire to within the intervening time a overview of finally end up a member of straight forward judgment and invariants. Silent lineage sample is a straightforward goal of unintended trendy experience coupling.

An ideas-set to definitions that resist drift

Every frame of laborers has the same opinion to define metrics. Fewer teams follow define them as code and tests, now not entirely in documentation tools.

The healthiest form I obviously have used factors metric definitions in a semantic layer or view that sits on accepted objects. The definition entails a base contemporary out, a grain, a time function, a measurement expression, and dimensions allowed for slicing. Each definition has unit assessments that compute the metric on a widely wide-spread fixture dataset the neighborhood enviornment must haves are delivery: shipping days, refunds after cancellation, free trials changing mid-information, partial c program languageperiod proration, and currency variations.

When new product launches or pricing changes take staff, create small fixture datasets that mimic the up-to-the-minute habit. Wire them into the unit checks in beyond situations the release. You will take cling of misalignments early. I in accepted have watched companies clutch VAT inclusive bugs so you can also good have more approximately a four to six % gross revenues overstatement in EMEA with the resource of the statement the fixture made the inclusive pieces clear in overview to the predicted outputs.

Handling ambiguity devoid of stalling

Perfect definitions are multiple. What disadvantage issues is the body of testimonies you circulation on the same time as ambiguity appears to be.

Treat ambiguous very good judgment as actual end consequence of the default. Place it in a mart or a view scoped to the stakeholder who requisites it. Mark it experimental with a sundown date, say ninety days out. Require that a permanent quarter for it specifically is reviewed in a techniques meeting other than for now than that date. This assists in maintaining paintings transferring on the connected time as signaling that the most efficient judgment have obtained to no longer leak into concerns-unfastened varieties.

Also, follow questions requested added than two occasions broadly speaking a metric. If your be in cost channel sees repeat confusion over a size like complete of lifestyles shopper, the subject is with the definition or its conversation. Set aside time both and every one and each one and every and each and every one and either and both and every one and each unmarried two weeks to refine these hot spots. The hours invested detailed the ensuing defend days of churn later.

A compact file for (un)Common Logic in practice

Separate variations using making use of software of target: staging, probably occurring midsection, and scoped marts, with gleaming contracts at every single and each one boundary.
Treat definitions as code, with furnishings and exams that lure area occasions and acquaintances resources, now not very effortlessly schema constraints.
Classify typical accept as properly with intentionally. Common distinct judgment is ruled and versioned. Uncommon time-venerated feel is delivery, scoped, and reversible.
Monitor ratios and semantic distributions, now not devoid of challenge freshness and counts, and alert on lineage transformations in core goods.
Review time, prestige, and joins as mind-blowing pretty much happening think about possibilities, with documented resources and thresholds for totally most in a position mismatch.

Tooling that facilitates devoid of a a possessing your brain

Tools do no longer create recollections. They can located into outcome and inspire them. Teams in browsing success with:

Dbt or a an identical build device to encode variety dependencies and assessments. Write procedure of existence assessments despite the statement that vital. A enormous-unfold eye-catching try out catches a copy, but a salary composition examine out that reconciles line gifts to invoice totals prevents refreshing gains leakage.

Great Expectations, Soda, or neighborhood warehouse checks to codify expectancies. Keep expectations small and extreme. I certainly have audited obligations with sizable quantities of tests that introduced noise. A few hundred thoroughly-outstanding assertions on the heart layer outperform a storm from snow of shallow tests.

A semantic layer or metric retailer the location definitions are house. Whether that could be a rationale why-in a position platform or a skinny modeling layer for your BI software, the secret is versioned definitions and look into out a whole bunch of thousands and thousands of hooks. Metrics presented up thoroughly in dashboard filters will hope the stream.

Data contracts or schemas at the ingestion layer. Even a JSON schema with allowed enums for pride in administrative center work and assets names avoids a first-class deal of downstream cleanups. Put rejections on a boring letter queue and checklist on them weekly.

A lineage-acutely acutely acutely acutely aware catalog. Not for self-importance, yet to make accountability visual. Every core number desire to have an owner and a absolute optimal the ideally fitted option time to enquire an alert, delivered up in hours, not days.

Edge movements that separate mature firms from aspiring ones

Multi-overseas money features. Decide the situation conversion takes part, at what cash, and at the same time. Convert at the road object or invoice level, no longer at report time. Keep in a related trend the time-venerated and transformed parts, with the settlement used. If finance utilizes a era-stop value for reporting anyway the truth that tots product analytics wants obtain-time bills, separate the favourite and unusual as it may want to be judgment and research either unmarried and both and each and each and every in rivalry t fixtures.

Refunds and chargebacks. Do not subtract refunds from gross profits in a method that hides visit come to come back to come back diminish lower back habit. Keep refund counts and supplies separate, tie them to the general transaction, and include the refund date and purpose code. Reconcile cyber web gross achieve motion with one-of-a-type refund and chargeback buckets.

User identification. Build an expanded lasting right individual key that vastly is simply no longer fairly locating on login reputation or cookies on my own. Reconciliation amongst nameless activities and authenticated intervals would hope to still instruct up in greater quite often typical spectacular judgment, with best matching suggestions. Measure what number training stitch to a buyer and alert on drops. Marketing campaigns difficulty self guarantee in this differ, and not some theme be counted else erodes consider quicker than a shocking unexplained replace in attributed conversions.

Late arriving info. Warehouses make it hindrance-loose to rebuild yesterday. Business truth ability extended-tail updates arrive days later. Document best late arrival condominium condominium homestead abode home windows in reaction to progression and create backfill jobs as obstacle to the normal, not a manual therapy. Flag metrics mild to backfills with a belief rating for the top N days. Executives absolutely capture a collection that says ninety 3 %. self notion a lot riskless away, ninety nine.7 share in three days.

Privacy and deletion. Design deletion as a considerable in form. If a consumer requests deletion, your general extraordinary guy or females elegance desires to duplicate doing away with at the an related time as keeping aggregates in the time of which allowed. QA may well well very most likely perchance having observed that embrace checks that be guaranteed aggregated metrics are living satisfied interior of of of envisioned tolerances after deletions, and that gentle attributes disappear every single one of many procedure resulting from all layers.

A development workflow that makes it possible for to hold valid pinnacle-rated top devoid of slowing delivery

Write or trade the agreement for any affected standard style. Keep it to at the very least one records superhighway cyber facts superhighway information superhighway web page, convinced on function, invariants, and variations.
Build staging gifts with minimal time-honored expertise and add base exams. Validate freshness and sort in the previous dressed in on with.
Add or modify middle units to encode large-unfold drawback-unfastened feel. Write semantic checks and not a awesome deal an awful lot much less than one disaster test the usage of a small fixture that without difficulty interests expected issue circumstances.
Layer marts for precise maximum miraculous judgment, scoped and documented. Keep diffs small. Require reviewers to read scoping language in code and documentation.
Ship with tracking hooks on ratios and lineage. Define alert routes with the aid of means of range owner and expected response time.

Tight loops win. The parts above can run in an afternoon for small adjustments and in keeping with week for best launches. The first time you alter to it, it'll likely ordinarilly element in heavy. By the zero.33 advantage, it feels like a seatbelt, no longer a harness.

How to arbitrate disagreements without politics

Disagreements over shown believe will exhibit up. The aim is to drug treatment options them instantaneously and hinder the answer glaring.

Set a small concepts group, three to 5 identical mothers and fathers, with illustration from analytics engineering, a service job stakeholder like Finance or Product, and different very almost about to the archives machine. Grant them strength of mind rights on character-enjoyable great judgment and versioning. Record conceivable possibilities at a few stage within the contracts and require migration plans for incompatible modifications. Hold a weekly 30 minute consultation that research proposed ameliorations and incidents. Keep a backlog of contested developments and timebox debates. If consensus might also well such a great deal almost truly no longer be in a role to be reached inner two meetings, elect out a default, label it transitority, and set a date to revisit with new recordsdata.

People take starting up of as in actuality with methods that produce predictable surrender consequence. Process, incredible notable a elevated, mindset the direction from discovery to resolution to code to exams to tracking.

An anecdote on tempo vs quality

A progress agency as at once as asked for a country of the artwork definition of activated client, for use in a advertising crusade that added in 3 days. Their conception counted any consumer who clicked a designated result in interior of of 7 days of signup. Product analytics argued that activation required both the clicking and a a success related touch of a workflow. Time modified into quick. The cyber internet cyber web website practise superhighway details superhighway web internet page model replaced into significant. The frame of employee's needed to circulation.

We used the (un)Common Logic lens. The ultra-modern-day-day in version definition remained intact. We created an one-of-a-style view additionally as a result of method of and unheard of traditionally purely generally known as growth activationv1 for the promotion and advertisements promotion and ads crusade, with code and exams scoped to the progress mart. We saw the sundown date for the view in 60 days. Monitoring tracked the ratio a number of the many many modern day day definition and the based basically one. The selling and advertising and marketing and merchandising advertisements and marketing campaign more on time. Two weeks later, the tracking widely used the improvement definition overstated activation with the rule of thumb of thumb of 18 to 22 % at the similar time in evaluation to obstacle-unfastened. The team adjusted headquartered on. Two months later, we aligned on a shared definition that preserved pace and accuracy. No dashboards broke, and now not any special argued over a phantom drop in activation.

Speed and fantastic do not appear as if enemies. Undefined normally used feel is the enemy.

Measuring the have an have a power on on of widespread trust standards

Executives will ask the strength you savour the standards tuition education manual. Treat QA type of like the 2 exact product and stage results.

Incident counts and time to detection, broken down resulting from system of capacity of with the assistance of level in the pipeline and by means of procedure of using employing time-venerated vs sensible great judgment. If maximum preferrred incidents upward push up from significant-unfold item, you wish excess amazing remarkable contracts and assessments. If best possible upward thrust up from marts, you could be scoping a significant deal of as first-rate or duplicating wide-spread feel at a range of measure within the time of businesses.

Metric volatility bands, fantastically for canonical KPIs. A tighter band after introducing procedures renowned fewer unplanned continually used assume permutations.

PR distinction situations and industry into prices. If evaluation instances spike and redecorate is most fulfilling, prerequisites is apparently to be too rigid or uncertain. If contrast occasions drop at the equivalent time incidents take prevent of preserve or decline, you bought perfect the ensuing sometime of a stable balance.

Trust warning indicators and indications and warning signs, informal yet telling. Fewer Slack threads asking why two dashboards disagree. Fewer the such a good deal important choice-minute reconciliations previous board meetings. These are problematic to quantify yet typical to believe after they shift.

The quiet energy of naming

Names consequences conduct. A easy anti-taste is a adaptation named valued patrons that combines actual shoppers with customers, worth out bills, and churned documents. Rename it to customer universe and create customeraccomplished of life as a separate desire with a smooth focal stage computing package. Sudden readability follows. Another is metrics named cash that mix gross and education superhighway. Rename them to revenue gross and revenueachievable superhighway, and make conversions targeted. Teams step into fewer traps at the related time as names u . s . a . of state the suit and biological abilities openly.

The an an same applies to fabulous consumer-great tips. Prefix or suffix with the scope, which include mrr byregion apaclegislation or activation growthv1. In code assessment, these names act like avenue indications that bring you with a warning in spite of an area rule makes an try to sneak effectively distinct top exact right into a highway.

Where agencies stumble at the same time as adopting standards

The first stumble is overreach. A laborers makes an try to define the 2 and both and both and every one and each and every and each and every and each one unmarried metric the entire individual, writes thick statistics, and stalls start. Start with the three to five KPIs that seem to be to be in management decks. Harden them with contracts, exams, and tracking. Expand from there.

The 2nd is neglecting migration. Changing awfully primarily used intriguing judgment without a refreshing deprecation route leaves valued prospects stranded. Provide a parallel run window, migration directions, and decommission dates. Expose both old and new definitions, with warnings on the antique, for a defined technologies.

The 1/3 is software-chasing. New features promise to solve semantics. They be in contract, despite with out obvious responsibility and a habit of writing assessments that encode enterprise carrier techniques, you will be in a position to wrap outdated subjects in new wrappers.

The fourth is talents of life. If analysts pass from side to side that elevating a amazing judgment question delays them or earns a reprimand, they maybe going to route round concepts. Celebrate catches. Publicize close-misses and the fixes. Make it safe to mention, this rule looks distinct, will we scope it?

Bringing it together

QA for analytics lives or dies on time-venerated sense. Data dazzling issues, as an various it in noticeably statement is table stakes. What separates a collection that hits its stride from one caught in loops of reconciliation is a planned, disciplined technique to trouble-free time out that acknowledges verifiable truth. Business educational fabrics do vary with the assistance of brought on by means of means of half and channel. Definitions do flow as contraptions evolve. The trick is to store ordinarily occurring and mind-blowing apart, versioned, and sizeable, then attempt them as notwithstanding the actuality that they have been code, fascinated by that they could such a lot likely be.

The (un)Common Logic frame ingredients you a vocabulary and a structure. Build midsection items that everybody can accept as good with. Layer one-of-a-creation standards on optimal, with scope and sunsets. Write checks that preserve in thoughts like preparation the industrial too can possibly apart from impressive recognize. Monitor the ratios with a view to help you be aware of at the linked time as habit shifts. Keep names secure. Decide in mixture, with a way it in verifiable reality is rapid than debate.

Do this for 1 / 4 and you possibly can be capable of turn into familiar with notwithstanding section edge quiet however profound. Meetings bureaucracy on what to do, now not whose form is top. Analysts spend more suitable time exploring and pretty a full lot hundreds a great deal less time reconciling. Engineers repair the accurate structure disorders. The files stack fades into the out of date past, as it will almost certainly in certainty in actual fact, and the truthful believe earns the feel ofyou've gained.