《风变编程服务使用协议》

第一条【协议充分了解】
请您（以下亦称“用户”）仔细阅读本协议全部条款，并确认您已完全了解本协议之规定。未成年人应当在监护人陪同和指导下阅读本协议，并在使用本协议项下服务前取得监护人的同意。请您审慎阅读并选择接受或不接受本协议。除非您接受本协议所有条款，否则您无权注册、登录或使用本协议所涉服务。您的注册、登录、使用等行为将视为对本协议的接受，并同意接受本协议各项条款的约束，本协议将构成您与风变科技(深圳)有限公司及其关联方（以下简称“风变编程”）之间关于您使用风变编程开发及运营的风变编程网站和软件应用（以下简称“‘风变编程’产品”）有约束力的法律文件。
本协议内容包括本协议正文及“风变编程”通过“风变编程”产品已经发布的或将来可能发布的各类规则。所有规则为本协议不可分割的组成部分，与协议正文具有同等法律效力。

第二条【用户充分授权】
您确认您依照中国法律法规及您所在地之法律及您自愿或应当接收其管辖之法律，有权签署并同意接受本协议的约束。如果您代表您的雇主或单位，您确认并保证您已获得签署本协议的充分授权。

第三条 【协议变更修改】
本协议（包括任何规则）如果变更，“风变编程”将尽可能提前在“风变编程”产品中发布通知。但您应了解并同意，“风变编程”有可能未经事先通知，而修改或变更本协议的全部或部分内容，您也应随时关注本协议是否有修改或变更之情形。如果本协议修改或变更，而您不能接受修改或变更后的内容，您应立即停止使用“风变编程”相关产品和服务。如果您继续使用“风变编程”相关产品和服务，则表明您完全接受并愿意遵守修改或变更后的本协议。无论您停止使用或继续使用“风变编程”相关产品和服务，您都承诺不因本协议修改或变更而向“风变编程”要求赔偿或补偿或提出其他权利主张。

第四条 【提供服务的方式】
“风变编程”是网络服务提供者，可针对不同传播渠道用户为其提供内容服务，前述传播渠道包括但不限于：
1） 通过“风变编程”的产品及服务；
2）“风变编程”提供的可以使用的其他传播渠道。 除免费服务外，“风变编程”可能提供部分收费服务。您可以通过付费方式购买收费服务，具体方式为：通过网上银行或其他“风变编程”产品内提供的其他付费途径支付一定数额的人民币，从而获得收费服务使用权限。对于收费服务，“风变编程”会在您使用之前给予明确的提示，只有根据提示确认同意按照前述支付方式支付费用并完成了支付行为，才能使用该等收费服务。支付行为的完成以银行或第三方支付平台生成“支付已完成”的确认通知为准。您应了解“风变编程”相关产品和服务可能对该等服务（含收费服务）进行更改、更新或升级，您同意不因“风变编程”更改、更新或升级其服务而向“风变编程”要求赔偿或补偿或提出其他权利主张。

第五条【善意合法使用】
您同意，遵守法律及本协议规定，秉承善意使用“风变编程”相关产品及服务，且保证：
1）不会利用“风变编程”相关产品及服务进行非法活动、或进行可能侵犯他人权利或损害他人利益的活动；不会以非法方式获取或利用“风变编程”产品及服务其他用户的信息。
2）不会以技术方式攻击或破坏或改变“风变编程”产品及服务的部分或全部、或干扰其运行；不会以非法方式获取或使用“风变编程”产品及服务的任何软件、代码、内容或其他技术或商业信息；不会对“风变编程”产品及服务运行的任何程序进行反向工程、反向编译、反向汇编或改写。
3）不会未经许可使用“风变编程”的网站名称、公司名称、商标、商业标识、网页版式或内容、或其他由“风变编程”享有知识产权或权利的信息或资料；不会侵犯“风变编程”的商标权、著作权、专利权、技术秘密、其他知识产权或其他合法权利或权益。不会以任何方式贬损“风变编程”的商业声誉。
4）不会利用“风变编程”进行其他违背公序良俗的活动。您使用“风变编程”产品及服务的记录，可能被保存作为对您不利的证据。“风变编程”也可能将您违反法律或侵犯第三方权利或权益的记录报告给行政主管部门、司法机关。
您理解并保证您在“风变编程”产品中上传、发布或传输的信息、内容（包括您的账户名称等信息）应当遵守宪法、法律和行政法规，不得含有以下内容，否则，“风变编程”将立即停止传输该信息，采取消除等处置措施，防止信息扩散，保存有关记录，并向有关主管部门报告：
1）反对宪法确定的基本原则的；
2）危害国家统一、主权和领土完整的；
3）泄露国家秘密、危害国家安全或者损害国家荣誉和利益的；
4）煽动民族仇恨、民族歧视，破坏民族团结，或者侵害民族风俗、习惯的；
5）宣扬邪教、迷信的；
6）扰乱社会秩序，破坏社会稳定的；
7）诱导未成年人违法犯罪和渲染暴力、色情、赌博、恐怖活动的；
8）侮辱或者诽谤他人，侵害公民个人名誉、隐私、知识产权和其他合法权益的；
9）危害社会公德，损害民族优秀文化传统的；
10）有关法律、行政法规和国家规定禁止的其他内容。
如果您上传、发布或传输的信息、内容含有以上违反法律法规的信息或内容，或者侵犯任何第三方的合法权益，您将直接承担以上导致的一切不利后果。如因此给“风变编程”造成不利后果的，您应负责消除影响，并且赔偿“风变编程”因此导致的一切损失，包括且不限于财产损害赔偿、名誉损害赔偿、律师费、交通费等因维权而产生的合理费用。

第六条 【注册用户账号】
您需要在“风变编程”产品中注册一个账号，并按照“风变编程”产品公示的要求进行账号设置，才可以使用该账号登录并使用“风变编程”相关产品和服务。您必须是拥有完全民事行为能力的民事主体，提供的信息必须真实合法，遵守法律法规及平台的管理规定；未成年人注册及使用“风变编程”产品须取得监护人的同意。如您违反上述注册规定，因此导致的法律后果、损失、责任将由您自行承担。您可依照“风变编程”产品公示的规则修改您的账号密码，但不可修改账号。 您注册账号时，应按照“风变编程”产品公示的提示及要求填写或提供资料、信息，并确保用户身份及信息的真实性、正确性及完整性；如果资料发生变化，您应及时更改。 您同意并承诺：
1）不故意冒用他人信息为您注册帐号；
2）不未经他人合法授权以他人名义注册帐号；
3）不使用色情、暴力或侮辱、诽谤他人等违反公序良俗的词语注册帐号。
如您违反前述规定，“风变编程”有权随时拒绝您使用该账号，或者限制您使用，或者注销该账号。

第七条 【用户账号使用】
在您完成账号注册后，“风变编程”将向您提供账号及初始登录方式，此后，您应自行保管及维持密码及账号的安全。您应自行使用该账号，并对该账号项下发生的一切行为及活动负完全的责任。您应了解，在账号和密码匹配时，“风变编程”无法对非法或未经您授权使用您账号及密码的行为作出甄别，因此，“风变编程”对任何使用您账号和密码登录并使用“风变编程”产品及服务的行为不承担任何责任。您同意并承诺：

1）当您的账号或密码遭到未获授权的使用，或者发生其他任何安全问题时，您会立即有效通知到“风变编程”；
2）当您每次上网或使用其他服务完毕后，会将有关账号等安全退出。“风变编程”有权根据自己的判定，在怀疑账号被不当使用时，拒绝账号使用或限制账号使用或注销该账号。除有使用期限的付费服务以外，如您连续180天未以账号登录“风变编程”产品，则“风变编程”有权根据自己的判定，注销该账号且无需事先向您发送通知。
您知悉并同意，您仅得依据本协议享有对“风变编程”产品账户及相关功能、服务的使用权，您仅能为自身学习、技术交流等非营利性目的使用账户及功能、服务；您不得出于任何非法或未经“风变编程”授权的目的使用账户或“风变编程”产品提供的功能、服务，包括但不限于，不得恶意注册账号（无论是否以营利为目的），不得利用账户或“风变编程”产品提供的功能、服务从事营利活动，不得以营利或非营利目的以任何方式（包括但不限于转让、借用、出租、共享等）向任何第三方提供账户使用权或“风变编程”产品提供的功能、服务或其任何部分，不得利用账户或“风变编程”产品提供的功能、服务从事任何违法或侵犯第三方知识产权或其他合法权益的活动，否则“风变编程”有权追究您的责任，且由此产生的一切责任均由您承担。

第八条 【发表言论】
用户在使用“风变编程”注册账号后，应遵守法律法规、社会主义制度、国家利益、公民合法权益、公共秩序、社会道德风尚和信息真实七条底线，不得在使用“风变编程”产品及服务时发表、传播、存储含有以下任何内容的信息：
1）违反宪法或法律法规规定的；
2）危害国家安全，泄露国家秘密，颠覆国家政权，破坏国家统一的；
3）损害国家荣誉和利益的，损害公共利益的；
4）煽动民族仇恨、民族歧视，破坏民族团结的；
5）破坏国家宗教政策，重扬邪教和封建迷信的；
6）散布谣言，扰乱社会秩序，破坏社会稳定的；
7）散布淫秽、色情、赌博、暴力、凶杀、恐怖或者教唆犯罪的；
8）侮辱或者诽谤他人，侵害他人合法权益的；
9）含有法律、行政法规禁止的其他内容的；
10）含有“风变编程”认为不适合在“风变编程”产品及服务中发表的其他内容的。

第九条 【个人信息及隐私条款】
“风变编程”收集、存储、使用、处理用户个人信息将遵守相关法律法规，收集、使用个人信息将遵循合法、正当、必要的原则。“风变编程”可能需要搜集能识别用户身份的个人信息以便可以在必要时联系用户，或为用户提供更好的使用体验。“个人信息”包括但不限于用户的姓名、性别、年龄、出生日期、联系电话、身份证号、地址、受教育情况、公司情况、所属行业、兴趣爱好、所处位置信息、个人说明、手机软硬件信息、手机识别码等。“风变编程”承诺，不以非法方式披露您依照“风变编程”规定的方式标明应予保密的用户个人信息。
您了解并同意，“风变编程”可依照法律或司法、行政机关的强制性命令披露您的个人信息且无需事先向您发出通知。您了解并同意，当您在“风变编程”产品中购买第三方商品或服务时，您的信息将被提供给该等商品或服务的提供方，以便其向您提供产品或服务，您与该等第三方商品或服务提供方之间的隐私保护事项由您与该等第三方商品或服务提供方自行解决，“风变编程”不因向该等第三方商品或服务提供方提供您的个人信息而向您承担任何责任。
您了解并同意，除为提供或改进“风变编程”产品及服务合理所需以外，“风变编程”在下述情形下可利用您的个人信息：
1）在紧急情况下，为维护用户及公众的权益。
2）为维护“风变编程”的著作权、商标权、专利权及其他任何合法权利或权益。
3）在进行促销或抽奖时，“风变编程”可能会与赞助商共享用户的个人信息，在这些情况下“风变编程”会在发送用户信息之前进行提示，并且用户可以通过明确表示不参与活动而终止传送过程。
4）为获取第三方数据而将用户信息与第三方数据匹配。
5）将用户数据用于统计，以便向未来的合作伙伴、广告商及其他第三方以及为了其他合法目的而描述“风变编程”的服务。

第十条 【知识产权】
“风变编程”对“风变编程”相关产品和服务及“风变编程”所使用的软件所包含的受知识产权法或其他法律保护的对象享有相应的权利。“风变编程”产品及服务的整体内容及其任何部分（包括但不仅限于网站设计、程序、代码、设计图样以及其他图样、产品及服务名称）的版权、商标权及其他知识产权归“风变编程”享有，任何人不得使用、复制或用作其他用途。经由“风变编程”产品及服务传送的文字、图片、音频、视频及其他内容，受到著作权法、商标法、专利法或其他法律的保护；未经相关权利人书面授权许可，您不得自行或许可第三方进行使用、修改、出租、售卖、传播、衍生其他作品或进行任何其他使用或处分。 “风变编程”对其专有内容、原创内容和其他通过授权取得的内容享有知识产权或相应权利。未经“风变编程”书面许可，任何单位和个人不得私自转载、传播和提供“风变编程”提供的服务或者实施任何其他侵犯“风变编程”知识产权的行为，否则，“风变编程”将依法追究相关法律责任。 为了促进知识的分享和传播，您同意将在“风变编程”产品上发表的全部内容（包括但不仅限于留言、评论和笔记），授予“风变编程”免费的、不可撤销的、非独家使用许可权，“风变编程”有权将该内容用于“风变编程”各种形态的产品和服务上，包括但不限于“风变编程”已经或未来发布的应用或其他互联网产品。您在“风变编程”产品上传或发表的内容，应保证您为著作权人或已取得合法授权，并且该内容不会侵犯任何第三方的合法权益。如果第三方提出关于著作权的异议，“风变编程”有权根据实际情况删除相关的内容，且有权追究用户的法律责任，给“风变编程”或任何第三方造成损失的，用户应负责全额赔偿。 “风变编程”有权对您发布的内容进行审核，有权根据相关证据结合《侵权责任法》、《信息网络传播权保护条例》等法律法规及“风变编程”公开的原则对侵权信息进行处理。

第十一条 【链接使用】
为方便您使用，“风变编程”可能会提供第三方国际互联网网站或资源链接。您应了解，除非另有声明，“风变编程”无法对第三方网站服务进行控制。您因下载或传播或使用或依赖上述网站或资源所生的损失或损害，应由您与第三方网站或相关责任方解决或处理，“风变编程”不承担任何责任。

第十二条 【有限责任及免责条款】
“风变编程”按照“风变编程”产品及服务的“现状”向用户提供产品及服务，“风变编程”不对“风变编程”产品及/或其功能、服务作出任何其他明示或暗示的保证。 对经由“风变编程”产品取得的任何产品、服务（含收费服务）或其他材料可能不符合您的期望；基于不同用户的个人特征和喜好，“风变编程”产品中可能存在一些让您不快、厌恶或难以接受的内容；“风变编程”不因前述情形向您承担任何责任。 用户在“风变编程”上进行的任何行为均是用户的个人行为，由于信息的海量性，“风变编程”无法对信息或内容的权属、合法性、合规性、真实性、科学性、完整权、有效性等进行审查。因用户行为或用户上传、存储、发布、传播的任何信息、内容等引发的任何争议、责任，或由此产生的任何直接、间接、偶然、特殊的损害，均由用户自行承担法律责任，“风变编程”不承担任何责任；如用户行为对“风变编程”造成损失或不良影响，“风变编程”保留追究相关用户法律责任的权利。 “风变编程”对用户所发布信息的删除或储存失败不承担任何法律责任。
不论在何种情况下，“风变编程”均不对由于互联网连接故障，电力故障，罢工，劳动争议，暴乱，起义，骚乱，火灾，洪水，风暴，爆炸，不可抗力，战争，政府行为，国际、国内法院的命令，黑客攻击，互联网病毒，网络运营商技术调整，政府临时管制或任何其他“风变编程”不能合理控制的原因而造成的“风变编程”产品不能访问、产品及服务中断、信息及数据的延误、停滞或错误，不能提供或延迟提供服务而承担责任。 不论是否可以预见，不论是源于何种形式的行为或不作为，“风变编程”不对因任何原因造成的任何特别的、间接的、惩罚性的、突发性的或其他任何损害（包括但不限于利润或其他可得利益的损失）承担责任。除“风变编程”因故意或重大过失损害用户的利益以外，任何情况下，“风变编程”向用户承担的赔偿、补偿或其他任何责任的限额将不超过该等用户就相关服务向“风变编程”支付的服务费用的数额。 基于互联网的开放性属性，用户知悉用户将信息、内容等上传到互联网上，可能会被其他组织或个人复制、转载、修改或进行其他使用（包括用于非法用途），用户必须充分意识到此类风险的存在。用户同意使用“风变编程”产品过程中所存在的上述风险将完全由用户自行承担，“风变编程”对此不承担任何责任。

第十三条 【“风变编程”的商业活动】
您了解并接受，“风变编程”为维护其运营所经营，可能进行下述商业活动：
1） 通过电子邮件、客户端、网页或其他合法方式向用户发送商品促销或其他相关商业信息。
2） 通过增值服务系统或其他方式向用户发送的相关服务信息或其他信息，其他信息包括但不限于通知信息、宣传信息、广告信息等。如您不同意收取此类信息，您可以选择退订相关信息。

第十四条 【非商业使用】
您同意，仅以非商业目的使用“风变编程”产品及服务；不对“风变编程”产品及服务的任何部分或全部进行商业性质利用、复制、拷贝、出售、调查、广告，或将“风变编程”产品及服务用于其他任何商业目的或商业性质的活动；但“风变编程”与您另行签订有协议或“风变编程”另行指定可供您使用的特定商业服务除外。

第十五条 【运行维护】
“风变编程”尽可能保证“风变编程”产品及服务的稳定运行。您应理解并同意，因系统维护、升级、调整，第三方原因如通讯设备故障、计算机病毒或黑客攻击、技术问题、网络、电脑故障、系统不稳定性、台风、地震、海啸、洪水、停电、战争、恐怖袭击、法律、政策、政府管制及其他各种不可抗力或“风变编程”无法合理控制的原因而引发的以下情况及因此导致的一切损失，“风变编程”不承担责任：
1）“风变编程”产品或服务无法使用或中断使用或无法完全适合用户的使用要求。
2）“风变编程”产品或服务受到干扰，无法及时、安全、可靠运行，或出现任何错误。
3）经由“风变编程”产品或服务取得的任何产品、服务（含收费服务）或其他材料不符合您的期望。
4）用户资料遭到未经授权的使用或修改。
“风变编程“对账号上所有服务将尽力维护其安全性、方便性和适用性，但对非因“风变编程”的故意或重大过失，用户资料遭到未经授权的使用或修改不承担责任。 “风变编程”不对您在本协议项下服务中相关数据的删除或储存失败负责。“风变编程”可以根据实际情况自行决定用户在本服务中数据的最长储存期限，并在服务器上为其分配数据最大存储空间等。如您停止使用“风变编程”产品，或“风变编程”所提供的产品或服务终止或中止，“风变编程”可以从服务器上永久地删除用户的数据。“风变编程”没有义务向用户返还任何数据。

第十六条 【服务终止】
本协议或本协议项下“风变编程”所提供的产品使用及服务（含收费服务）可在下述情形下部分或全部中止或终止：
1） 因法律规定，或“风变编程”服从行政命令或司法判令的要求。
2） 用户违反本协议。
3） “风变编程”认为应予终止的其他情形。
在一般情形下，“风变编程”会提前按照您提供的联系方式以电子邮件或短信或其他电子方式通知您产品使用及服务将中止或终止。 您应了解并同意，在紧急情况或特殊下，“风变编程”可不经通知即中止或终止产品使用及服务。在本协议或本协议项下“风变编程”提供的产品使用及服务（含收费服务）中止或终止时，“风变编程”有权 ：
1）拒绝用户登录“风变编程”产品及服务；
2）删除用户信息；
3）删除用户购买的内容。
您同意，如因“风变编程”原因导致付费产品在到期前终止使用，“风变编程”仅需按照相关产品规则向您返还未到期部分对应的价款 ，“风变编程”无需承担其他责任或义务，除上述情形外，“风变编程”无需因本协议或本协议项下“风变编程”所提供的产品使用及服务中止或终止，而向您作出赔偿或补偿或承担任何其他责任。

第十七条 【法律适用】
您在使用“风变编程”产品及服务时，应遵守中华人民共和国法律、及您所在地之法律、及您自愿或应当接收其管辖之法律。本协议的解释及履行应按照中华人民共和国（为本协议之目的，不包括港澳台地区）法律进行。

第十八条 【法律责任】
您应就所上传的内容承担全部法律责任；无论前述责任是因侵犯第三方权利所引起的、或因您违反与第三方或本协议引起的、或因您违反法律引起的；前述责任包括对“风变编程”或第三方所承担的民事责任、或行政机关要求承担的行政责任或刑事责任。 您同意，如因您违反法律规定或本协议规定给“风变编程”造成损失，您将充分赔偿“风变编程”所遭受的损失、包括其直接损失、间接损失、预期利益损失等一切损失。 您确认，如因第三方向您主张权利而导致您承担责任，或您承担行政责任或刑事责任的，“风变编程”无需向您承担任何责任。如您应向“风变编程”承担责任，前述责任不因本协议或本协议项下的服务被终止而免除。 如您在使用“风变编程”产品及服务的过程中发生任何违反法律法规或本协议的行为，“风变编程”有权根据行为过错、性质、后果等采取相应措施，包括但不限于删除相关内容、暂停账户使用、解除本协议、终止产品使用及服务、追索赔偿等，由此造成的一切风险、损失及责任将由您自行承担。

第十九条 【协议构成】
“风变编程”可能不时通过“风变编程”产品发布针对用户的相关协议或规则，该等相关协议或规则作为对本协议条款的补充或修改而成为本协议的一部分。请您及时关注并阅读相关协议或规则。

第二十条 【争议解决】
您和“风变编程”均同意，因本协议解释或执行引起的任何争议，双方应首先友好协商解决。协商不成时，争议将提交“风变编程”所在地法院诉讼解决。 请您再次确认您已全部阅读并充分理解上述协议。 如有任何疑问，可通过客服邮箱【code@forchange.cn】与“风变编程”取得联系。

首先下载三个软件
– 增值税发票税控开票软件
– eTax@SH 3
– 自然人税收管理系统扣缴客户端

报税顺序：

月头发票上报汇总

国税局报税，地址：

https://etax.shanghai.chinatax.gov.cn/wszx-web/bszm/apps/views/beforeLogin/indexBefore/pageIndex.html#/

每个月8号之前报上个月的个人税

每月15号之前反写清盘，用于开发票

每季度8号之前报小规模企业税

注意：季度票不超过9万免税
每季度打印一次季度汇总表
税控盘每个月15号前都要上报汇总和反写清卡

~~~~~~~~~~~~~~~~~~~
小规模企业季报清卡步骤（1~15号内按顺序完成）
申报月(1,4,7,10月)操作步骤：
发票同步->上报汇总(个人所得税和企业税，资产负债表，现金流量表，利润表)->申报->反写
非申报月操作步骤：(2,3,5,6,8,9,11,12)
发票同步->上报汇总(个人所得税和企业税)->反写

Measuring the User Experience on a Large Scale:
User-Centered Metrics for Web Applications
Kerry Rodden, Hilary Hutchinson, and Xin Fu
Google
1600 Amphitheatre Parkway, Mountain View, CA 94043, USA
{krodden, hhutchinson, xfu}@google.com
ABSTRACT
More and more products and services are being deployed
on the web, and this presents new challenges and
opportunities for measurement of user experience on a large
scale. There is a strong need for user-centered metrics for
web applications, which can be used to measure progress
towards key goals, and drive product decisions. In this
note, we describe the HEART framework for user-centered
metrics, as well as a process for mapping product goals to
metrics. We include practical examples of how HEART
metrics have helped product teams make decisions that are
both data-driven and user-centered. The framework and
process have generalized to enough of our company’s own
products that we are confident that teams in other
organizations will be able to reuse or adapt them. We also
hope to encourage more research into metrics based on
large-scale behavioral data.
Author Keywords
Metrics, web analytics, web applications, log analysis.
ACM Classification Keywords
H.5.2 [Information interfaces and presentation]: User
Interfaces—benchmarking, evaluation/methodology.
General Terms
Experimentation, Human Factors, Measurement.
INTRODUCTION
Advances in web technology have enabled more
applications and services to become web-based and
increasingly interactive. It is now possible for users to do a
wide range of common tasks “in the cloud”, including those
that were previously restricted to native client applications
(e.g. word processing, editing photos). For user experience
professionals, one of the key implications of this shift is the
ability to use web server log data to track product usage on
a large scale. With additional instrumentation, it is also
possible to run controlled experiments (A/B tests) that
compare interface alternatives. But on what criteria should
they be compared, from a user-centered perspective? How
should we scale up the familiar metrics of user experience,
and what new opportunities exist?
In the CHI community, there is already an established
practice of measuring attitudinal data (such as satisfaction)
on both a small scale (in the lab) and a large scale (via
surveys). However, in terms of behavioral data, the
established measurements are mostly small-scale, and
gathered with stopwatches and checklists as part of lab
experiments, e.g. effectiveness (task completion rate, error
rate) and efficiency (time-on-task) [13].
A key missing piece in CHI research is user experience
metrics based on large-scale behavioral data. The web
analytics community has been working to shift the focus
from simple page hit counts to key performance indicators.
However, the typical motivations in that community are
still largely business-centered rather than user-centered.
Web analytics packages provide off-the-shelf metrics
solutions that may be too generic to address user experience
questions, or too specific to the e-commerce context to be
useful for the wide range of applications and interactions
that are possible on the web.
We have created a framework and process for defining
large-scale user-centered metrics, both attitudinal and
behavioral. We generalized these from our experiences of
working at a large company whose products cover a wide
range of categories (both consumer-oriented and businessoriented),
are almost all web-based, and have millions of
users each. We have found that the framework and process
have been applicable to, and useful for, enough of our
company’s own products that we are confident that teams in
other organizations will be able to reuse or adapt them
successfully. We also hope to encourage more research into
metrics based on large-scale behavioral data, in particular.
RELATED WORK
Many tools have become available in recent years to help
with the tracking and analysis of metrics for web sites and
applications. Commercial and freely available analytics
© ACM, 2010. This is the author’s version of the work. It is posted here
by permission of ACM for your personal use. Not for redistribution. The
definitive version was published in the Proceedings of CHI 2010, April
10–15, 2010, Atlanta, Georgia, USA.
packages [5,11] provide off the shelf solutions. Custom
analysis of large-scale log data is made easier via modern
distributed systems [4,8] and specialized programming
languages [e.g. 12]. Web usage mining techniques can be
used to segment visitors to a site according to their behavior
[3]. Multiple vendors support rapid deployment and
analysis of user surveys, and some also provide software for
large-scale remote usability or benchmarking tests [e.g. 14].
A large body of work exists on the proper design and
analysis of controlled A/B tests [e.g. 10] where two similar
populations of users are given different user interfaces, and
their responses can be rigorously measured and compared.
Despite this progress, it can still be challenging to use these
tools effectively. Standard web analytics metrics may be
too generic to apply to a particular product goal or research
question. The sheer amount of data available can be
overwhelming, and it is necessary to scope out exactly what
to look for, and what actions will be taken as a result.
Several experts suggest a best practice of focusing on a
small number of key business or user goals, and using
metrics to help track progress towards them [2, 9, 10]. We
share this philosophy, but have found that this is often
easier said than done. Product teams have not always
agreed on or clearly articulated their goals, which makes
defining related metrics difficult.
It is clear that metrics should not stand alone. They should
be triangulated with findings from other sources, such as
usability studies and field studies [6,9], which leads to
better decision-making [15]. Also, they are primarily useful
for evaluation of launched products, and are not a substitute
for early or formative user research. We sought to create a
framework that would combine large-scale attitudinal and
behavioral data, and complement, not replace, existing user
experience research methods in use at our company.
PULSE METRICS
The most commonly used large-scale metrics are focused
on business or technical aspects of a product, and they (or
similar variations) are widely used by many organizations
to track overall product health. We call these PULSE
metrics: Page views, Uptime, Latency, Seven-day active
users (i.e. the number of unique users who used the product
at least once in the last week), and Earnings.
These metrics are all extremely important, and are related to
user experience – for example, a product that has a lot of
outages (low uptime) or is very slow (high latency) is
unlikely to attract users. An e-commerce site whose
purchasing flow has too many steps is likely to earn less
money. A product with an excellent user experience is
more likely to see increases in page views and unique users.
However, these are all either very low-level or indirect
metrics of user experience, making them problematic when
used to evaluate the impact of user interface changes. They
may also have ambiguous interpretation – for example, a
rise in page views for a particular feature may occur
because the feature is genuinely popular, or because a
confusing interface leads users to get lost in it, clicking
around to figure out how to escape. A change that brings in
more revenue in the short term may result in a poorer user
experience that drives away users in the longer term.
A count of unique users over a given time period, such as
seven-day active users, is commonly used as a metric of
user experience. It measures the overall volume of the user
base, but gives no insight into the users’ level of
commitment to the product, such as how frequently each of
them visited during the seven days. It also does not
differentiate between new users and returning users. In a
worst-case retention scenario of 100% turnover in the user
base from week to week, the count of seven-day active
users could still increase, in theory.
HEART METRICS
Based on the shortcomings we saw in PULSE, both for
measuring user experience quality, and providing
actionable data, we created a complementary metrics
framework, HEART: Happiness, Engagement, Adoption,
Retention, and Task success. These are categories, from
which teams can then define the specific metrics that they
will use to track progress towards goals. The Happiness and
Task Success categories are generalized from existing user
experience metrics: Happiness incorporates satisfaction,
and Task Success incorporates both effectiveness and
efficiency. Engagement, Adoption, and Retention are new
categories, made possible by large-scale behavioral data.
The framework originated from our experiences of working
with teams to create and track user-centered metrics for
their products. We started to see patterns in the types of
metrics we were using or suggesting, and realized that
generalizing these into a framework would make the
principles more memorable, and usable by other teams.
It is not always appropriate to employ metrics from every
category, but referring to the framework helps to make an
explicit decision about including or excluding a particular
category. For example, Engagement may not be meaningful
in an enterprise context, if users are expected to use the
product as part of their work. In this case a team may
choose to focus more on Happiness or Task Success. But it
may still be meaningful to consider Engagement at a feature
level, rather than the overall product level.
Happiness
We use the term “Happiness” to describe metrics that are
attitudinal in nature. These relate to subjective aspects of
user experience, like satisfaction, visual appeal, likelihood
to recommend, and perceived ease of use. With a general,
well-designed survey, it is possible to track the same
metrics over time to see progress as changes are made.
For example, our site has a personalized homepage,
iGoogle. The team tracks a number of metrics via a weekly
in-product survey, to understand the impact of changes and
new features. After launching a major redesign, they saw
an initial decline in their user satisfaction metric (measured
on a 7-point bipolar scale). However, this metric recovered
over time, indicating that change aversion was probably the
cause, and that once users got used to the new design, they
liked it. With this information, the team was able to make a
more confident decision to keep the new design.
Engagement
Engagement is the user’s level of involvement with a
product; in the metrics context, the term is normally used to
refer to behavioral proxies such as the frequency, intensity,
or depth of interaction over some time period. Examples
might include the number of visits per user per week, or the
number of photos uploaded per user per day. It is generally
more useful to report Engagement metrics as an average per
user, rather than as a total count – because an increase in
the total could be a result of more users, not more usage.
For example, the Gmail team wanted to understand more
about the level of engagement of their users than was
possible with the PULSE metric of seven-day active users
(which simply counts how many users visited the product at
least once within the last week). With the reasoning that
engaged users should check their email account regularly,
as part of their daily routine, our chosen metric was the
percentage of active users who visited the product on five
or more days during the last week. We also found that this
was strongly predictive of longer-term retention, and
therefore could be used as a bellwether for that metric.
Adoption and Retention
Adoption and Retention metrics can be used to provide
stronger insight into counts of the number of unique users
in a given time period (e.g. seven-day active users),
addressing the problem of distinguishing new users from
existing users. Adoption metrics track how many new users
start using a product during a given time period (for
example, the number of accounts created in the last seven
days), and Retention metrics track how many of the users
from a given time period are still present in some later time
period (for example, the percentage of seven-day active
users in a given week who are still seven-day active three
months later). What counts as “using” a product can vary
depending on its nature and goals. In some cases just
visiting its site might count. In others, you might want to
count a visitor as having adopted a product only if they
have successfully completed a key task, like creating an
account. Like Engagement, Retention can be measured over
different time periods – for some products you might want
to look at week-to-week Retention, while for others
monthly or 90-day might be more appropriate. Adoption
and Retention tend to be especially useful for new products
and features, or those undergoing redesigns; for more
established products they tend to stabilize over time, except
for seasonal changes or external events.
For example, during the stock market meltdown in
September 2008, Google Finance had a surge in both page
views and seven-day active users. However, these metrics
did not indicate whether the surge was driven by new users
interested in the crisis, or existing users panic-checking
their investments. Without knowing who was making more
visits, it was difficult to know if or how to change the site.
We looked at Adoption and Retention metrics to separate
these user types, and examine the rate at which new users
were choosing to continue using the site. The team was
able to use this information to better understand the
opportunities presented by event-driven traffic spikes.
Task Success
Finally, the “Task Success” category encompasses several
traditional behavioral metrics of user experience, such as
efficiency (e.g. time to complete a task), effectiveness (e.g.
percent of tasks completed), and error rate. One way to
measure these on a large scale is via a remote usability or
benchmarking study, where users can be assigned specific
tasks. With web server log file data, it can be difficult to
know which task the user was trying to accomplish,
depending on the nature of the site. If an optimal path exists
for a particular task (e.g. a multi-step sign-up process) it is
possible to measure how closely users follow it [7].
For example, Google Maps used to have two different types
of search boxes – a dual box for local search, where users
could enter the “what” and “where” aspects separately (e.g.
[pizza][nyc]) and a single search box that handled all kinds
of searches (including local searches such as [pizza nyc], or
[nyc] followed by [pizza]). The team believed that the
single-box approach was simplest and most efficient, so, in
an A/B test, they tried a version that offered only the single
box. They compared error rates in the two versions, finding
that users in the single-box condition were able to
successfully adapt their search strategies. This assured the
team that they could remove the dual box for all users.
GOALS – SIGNALS – METRICS
No matter how user-centered a metric is, it is unlikely to be
useful in practice unless it explicitly relates to a goal, and
can be used to track progress towards that goal. We
developed a simple process that steps teams through
articulating the goals of a product or feature, then
identifying signals that indicate success, and finally
building specific metrics to track on a dashboard.
Goals
The first step is identifying the goals of the product or
feature, especially in terms of user experience. What tasks
do users need to accomplish? What is the redesign trying to
achieve? Use the HEART framework to prompt articulation
of goals (e.g. is it more important to attract new users, or to
encourage existing users to become more engaged?). Some
tips that we have found helpful:
• Different team members may disagree about what the
project goals are. This process provides a great
opportunity to collect all the different ideas and work
towards consensus (and buy-in for the chosen metrics).
• Goals for the success of a particular project or feature
may be different from those for the product as a whole.
• Do not get too distracted at this stage by worrying
about whether or how it will be possible to find
relevant signals or metrics.
Signals
Next, think about how success or failure in the goals might
manifest itself in user behavior or attitudes. What actions
would indicate the goal had been met? What feelings or
perceptions would correlate with success or failure? At this
stage you should consider what your data sources for these
signals will be, e.g. for logs-based behavioral signals, are
the relevant actions currently being logged, or could they
be? How will you gather attitudinal signals – could you
deploy a survey on a regular basis? Logs and surveys are
the two signal sources we have used most often, but there
are other possibilities (e.g. using a panel of judges to
provide ratings). Some tips that we have found helpful:
• Choose signals that are sensitive and specific to the
goal – they should move only when the user experience
is better or worse, not for other, unrelated reasons.
• Sometimes failure is easier to identify than success (e.g
abandonment of a task, “undo” events [1], frustration).
Metrics
Finally, think about how these signals can be translated into
specific metrics, suitable for tracking over time on a
dashboard. Some tips that we have found helpful:
• Raw counts will go up as your user base grows, and
need to be normalized; ratios, percentages, or averages
per user are often more useful.
• There are many challenges in ensuring accuracy of
metrics based on web logs, such as filtering out traffic
from automated sources (e.g. crawlers, spammers), and
ensuring that all of the important user actions are being
logged (which may not happen by default, especially in
the case of AJAX or Flash-based applications).
• If it is important to be able to compare your project or
product to others, you may need to track additional
metrics from the standard set used by those products.
CONCLUSIONS
We have spent several years working on the problem of
developing large-scale user-centered product metrics. This
has led to our development of the HEART framework and
the Goals-Signals-Metrics process, which we have applied
to more than 20 different products and projects from a wide
variety of areas within Google. We have described several
examples in this note of how the resulting metrics have
helped product teams make decisions that are both datadriven
and user-centered. We have also found that the
framework and process are extremely helpful for focusing
discussions with teams. They have generalized to enough of
our company’s own products that we are confident that
teams in other organizations will be able to reuse or adapt
them successfully. We have fine-tuned both the framework
and process over more than a year of use, but the core of
each has remained stable, and the framework’s categories
are comprehensive enough to fit new metrics ideas into.
Because large-scale behavioral metrics are relatively new,
we hope to see more CHI research on this topic – for
example, to establish which metrics in each category give
the most accurate reflection of user experience quality.
ACKNOWLEDGMENTS
Thanks to Aaron Sedley, Geoff Davis, and Melanie Kellar
for contributing to HEART, and Patrick Larvie for support.
REFERENCES
1. Akers, D. et al. (2009). Undo and Erase Events as
Indicators of Usability Problems. Proc of CHI 2009,
ACM Press, pp. 659-668.
2. Burby, J. & Atchison, S. (2007). Actionable Web
Analytics. Indianapolis: Wiley Publishing, Inc.
3. Chi, E. et al. (2002). LumberJack: Intelligent Discovery
and Analysis of Web User Traffic Composition. Proc of
WebKDD 2002, ACM Press, pp. 1-15.
4. Dean, J. & Ghemawat, S. (2008). MapReduce:
Simplified Data Processing on Large Clusters.
Communications of the ACM, 51 (1), pp. 107-113.
5. Google Analytics: http://www.google.com/analytics
6. Grimes, C. et al. (2007). Query Logs Alone are not
Enough. Proc of WWW 07 Workshop on Query Log
Analysis: http://querylogs2007.webir.org
7. Gwizdka, J. & Spence, I. (2007). Implicit Measures of
Lostness and Success in Web Navigation. Interacting
with Computers 19(3), pp. 357-369.
8. Hadoop: http://hadoop.apache.org/core
9. Kaushik, A. (2007). Web Analytics: An Hour a Day.
Indianapolis: Wiley Publishing, Inc.
10. Kohavi, R. et al. (2007). Practical Guide to Controlled
Experiments on the Web. Proc of KDD 07, ACM Press,
pp. 959-967.
11. Omniture: http://www.omniture.com
12. Pike, R. et al. (2005). Interpreting the Data: Parallel
Analysis with Sawzall. Scientific Programming (13), pp.
277-298.
13. Tullis, T. & Albert, W. (2008). Measuring the User
Experience. Burlington: Morgan Kaufmann.
14. UserZoom: http://www.userzoom.com
15. Weischedel, B. & Huizingh, E. (2006). Website
Optimization with Web Metrics: A Case Study. Proc of
ICEC 06, ACM Press, pp. 463-470.